P&ID tag extraction and verification

From drawings to a tag inventory

Process and automation engineers spend hours reading piping and instrumentation diagrams, or P&IDs, to prepare tag inventories. They find symbols, read tag numbers and functions, enter the values, then check the result. In existing facilities, complex drawings and frequent changes add more work and more chances for mistakes.

I am building an application that uses AI to prepare the first inventory from a PDF. Engineers check it against the drawing, correct missing or incorrect information, and export their reviewed data.

My contribution

Since January 2026, I have been rebuilding an internal extraction application using ideas from an inherited prototype, without reusing its code. I am responsible for the architecture, cloud infrastructure, backend, frontend and model-related work. I work with stakeholders to understand what the application needs to do.

Making review worth the effort

Fast extraction only helps if checking and correcting its output takes less time than preparing the inventory manually. The measure of success is the total time needed to reach an agreed quality standard, including review and corrections. That means looking at missed tags and correction effort as well as model accuracy. Time savings have not yet been validated.

Finding small tags in large drawings

Large PDF pages create a problem for detection: a high-resolution image preserves small tags but is too large to process as one model input. Shrinking it makes those tags harder to find.

I explored sliced inference with SAHI. The current detector processes overlapping tiles and merges the detections back onto the page. This keeps the original detail available without sending the whole image through the model at once. Validation across drawing vendors is still limited.

From detection to engineer review

The application renders PDFs, detects tags, classifies image crops and extracts text before presenting the results for review. Python and FastAPI connect the services. PyMuPDF renders the pages; Detectron2 and SAHI handle detection; a Swin classifier runs through PyTorch and timm; Azure OpenAI supports semantic extraction.

The review interface uses JavaScript and Konva. Engineers can check results on the drawing, correct fields, add or remove annotations, adjust boxes, leave notes and undo changes. They can export the reviewed inventory as CSV, XLSX or HTML.

Keeping a record of changes

I limited concurrent extraction requests to reduce pressure on the model service. That limit currently applies within one process. I also keep the original predictions alongside engineer corrections so later evaluation can distinguish model output from manual edits.

Newer PostgreSQL services handle identity, ownership, versions and audit records. Large files are stored separately and kept unchanged. The current review interface still stores its state in JSON. Implemented backend services can freeze a review package, check file hashes, record where annotations came from and control access to private files.

Engineers can export their corrected inventories without agreeing to let their annotations be used for model training.

What is ready and what is next

The application is under active development. Validation across drawing vendors is still limited. Time savings and error reduction have not been verified. I am continuing development toward wider internal use, focusing on reliable extraction and a review process that keeps a clear record of changes.

Future model improvement would use annotations with permission and independent verification. Automated retraining and wider rollout are planned, not completed.