Revolutionizing Steel Plant Operations

SteelEye is an intelligent ladle tracking system designed to revolutionize steel plant operations by automating the detection and monitoring of ladles throughout their lifecycle. By leveraging computer vision and deep learning, SteelEye eliminates manual tracking inefficiencies and provides real-time operational insights.

The system detects handwritten ladle numbers using a fine-tuned YOLOv8 model, maps them to critical parameters like temperature and circulation time, and visualizes comprehensive analytics through an intuitive web dashboard. An integrated RAG-based SQL chatbot enables operators to query ladle data conversationally, making tracking accessible and actionable.

• Automated Ladle Number Detection with YOLOv8
• Real-Time Analytics and monitoring
• History Tracking and lifecycle management
• Intelligent SQL Chatbot with RAG technology
• Database Integration with FastAPI and MySQL

Current Status: Prototype successfully tested with single-camera deployment; multi-camera setup will be tested on site soon

Why Ladle Tracking Matters

In steel manufacturing, ladles transport molten steel from converters to casting mills. Maintaining optimal temperature (superheat) throughout this journey is critical for casting quality. However, traditional tracking methods face several challenges:

• Manual Monitoring: Inefficient, error-prone, unreliable and labor-intensive
• Temperature Loss: Extended transit times reduce superheat, affecting product quality
• Ladle Life: Excessive heat exposure erodes linings, shortening ladle life
• Operational Inefficiency: Lack of real-time location and history data hampers planning

Impact on Operations

Without automated tracking, operators cannot:

• Assess ladle health before converter processes
• Minimize temperature loss in a cycle through optimized scheduling
• Monitor ladle circulation and turnaround times
• Access historical maintenance and repair data instantly
• Find time-consuming bottlenecks in the operation and investigate other issues to reduce temperature loss etc.

Key Innovation

SteelEye introduces a vision-based ladle tracking system that uses multiple camera feeds to detect ladle numbers (handwritten or printed) and maps them to teh camera location in the plant, hence tracking the ladle from unit to unit.

Other solutions like GPS and RFID come with problems of device cost, installation cost, maintenance cost since the steel ladle is at high temperature, for any device to be put on it, it will need a cooling system, and frequent charging.

Hence, our solution is to put up a camera unit at every unit/sub-unit in the plant where ladle would arrive for further processing and detect the ladle number hence track each ladle's location.

Key Parameters Tracked

1. 1. Ladle Circulation Time – Total time from tapping (loading metal into the ladle from the BOF Converter) to return
2. 2. Ladle Life – Cumulative usage and remaining lifespan
3. 3. Ladle Return Time – Duration from casting completion to availability
4. 4. Ladle Turnaround Time – Time between consecutive uses
5. 5. Ladle Temperature & Superheat – Real-time thermal monitoring

System Capabilities

• Automated Detection: Fine-tuned YOLOv8 model detects ladle numbers with high accuracy
• Real-Time Analytics: Live dashboard displays ladle locations, temperature, and workflow status
• Historical Tracking: Complete ladle history, maintenance records, and lifecycle analytics
• Intelligent Chatbot: RAG-powered conversational interface for querying ladle data
• Database Integration: FastAPI backend with MySQL database for robust data management

Technology Stack

• Computer Vision: YOLOv8 object detection model fine-tuned on 200+ images of steel plant ladles
• Backend: FastAPI (Python) with MySQL database integration
• Frontend: React.js for dynamic, responsive user interface
• AI Assistant: RAG chatbot with SQL query generation for conversational data access
• Deployment: Multi-camera support with centralized processing

Model Performance

Our YOLOv8 model was fine-tuned on a custom dataset that we captured and labeled at Rashtriya Ispat Nigam Limited (Vizag Steel). Every ladle and handwritten number was annotated in-house before training, and we incorporated diverse augmentation strategies to handle challenging conditions such as:

• Varying lighting and angles
• Motion blur and occlusion
• Handwritten number variations
• Industrial environmental noise

Benefits to Steel Plant Operations

• Reduced Manual Labor: Eliminates hand-written logs and duplicate data entry
• Energy and Time Efficiency Since temperature loss will be minimal and reheating would not be needed before casting, time and energt are saved
• Temperature Management: Minimizes superheat loss by poitning out inefficiencies
• Extended Ladle Life: Protects refractory linings by tracking cumulative exposure
• Enhanced Transparency: Real-time visibility for shop-floor operators and plant leadership
• Data-Driven Decisions: Historical analytics unlock predictive maintenance insights
• Improved Productivity: Faster turnaround times by enabling proactive planning

Comprehensive Dashboard

The SteelEye web application provides a centralized control center for monitoring all ladle operations in real-time. Key features include:

• Live Tracking: Real-time ladle location visualization across plant zones
• Temperature Monitoring: Current and historical temperature data with alerts
• Analytics Dashboard: Circulation time, turnaround metrics, and efficiency insights
• Maintenance History: Complete lifecycle records and repair schedules
• RAG Chatbot: Natural language queries for instant data retrieval