From geometry to reasoning

How Falkor turns engineering data into a world AI agents can see, simulate, and reason over, with NVIDIA Omniverse.

At NVIDIA GTC Berlin, Falkor, a KONGSBERG company, demonstrates how AI agents can see and work within industrial facilities, with NVIDIA Omniverse libraries composed into the Falkor Platform and brought to people through Falkor Surface. This post shows what that means for the people who plan, inspect, and investigate the work.

What makes this footage interesting goes much deeper than what you immediately see: a dynamic simulation of an industrial facility, using a SimReady OpenUSD representation derived directly and automatically from the facility's engineering CAD. Individual geometric objects are linked to their corresponding asset tags, while agent-driven semantic mining enriches the scene with information from the facility's engineering data. Every light visible in the scene originates from a fixture documented in that data, and its light is physically simulated. When the underlying CAD changes, the scene can be regenerated by rerunning the pipeline. Geometry becomes semantics. Semantics drive behavior. Agents can reason over the resulting simulated evidence.

What the asset already knows

Every asset runs inside an operating envelope, and every intervention has to fit a window: a weather window offshore, a maintenance window, a permit window. Alarms and technical monitoring guard the edges of the envelope. The harder work happens inside it: noticing a change that is not an alarm, tracing it through the process, and working out what caused it. That is where the exceptions worth acting on are found.

The stakes are measured in deferment: production lost or delayed. Much of the work behind it is manual and repeated, and time spent monitoring and interpreting signals is time not spent on production optimization, where a fraction of a percent adds up to significant value over an asset's life. Inspection and maintenance absorb thousands of hours on a large facility, much of it because it is hard for people to scope exactly what should be inspected, where, and when.

Behind all of this sits the energy trilemma: supply that must stay secure, energy that must stay affordable, and emissions that must come down, all from the same assets. It is not solved once. It is balanced, every day.

The data needed to strike that balance mostly exists: the CAD model, the tag register, the maintenance history, the process historian, the inspection records. But industrial data is permanently incomplete. It sits in systems that were never designed to be read together, and the as-built diverges from the as-designed on day one and keeps diverging. An experienced engineer does not stop working because the documentation is wrong. They reason around it. Software now has to do the same, against the physical plant: where things are, what surrounds them, what they connect to, and what happens when something changes. Data quality becomes an output of that loop rather than a precondition for entering it.

AI agents shorten the distance between a complex question and a decision someone is willing to sign off on. The decision stays with the person accountable for it, whether that is a work permit, an inspection plan, or a lift plan. What moves is how fast the evidence behind it can be assembled.

Two things make the rest possible

Falkor's Digital Twin has connected industrial identity, engineering information, operational data, documents, geometry, and trusted models for years, and its AI agents already work across that context. Agents can interpret a person's objective, call specialized tools, invoke simulations, and collaborate with other agents. The Digital Twin has long made geometry useful to people. The emerging question is how geometry becomes useful to intelligent machines.

The first is that the asset became legible. A CAD model may contain millions of shapes and loosely structured attributes. An experienced engineer sees pumps, separators, valves, pipe systems, access routes, and functional relationships. Agents orient themselves in raw engineering data, learning a facility's own vocabulary from its structure and attributes, without imposing a standard, so the plant can answer questions about what it contains. Falkor calls this semantic mining. It represents years of work, and everything below depends on it.

The second is that the plant became observable. A physically based render obeys the physics of seeing: what is visible from where, what blocks what, how far light reaches, and how big things are relative to each other. The image is a consequence of what the engineering data says, which makes it something an agent can reason over. One example: light a facility using only the fixtures its own data documents, and you can ask whether a work area is lit well enough to run a job at night. Where an area stays dark, that points to a possible lighting gap or missing fixture data, and either is worth investigating. Rendering gives the agent eyes. The same render serves two readers. A person walks the asset at any hour from anywhere. An agent requests a view, identifies the equipment behind a given pixel, changes the scene, and looks again.

Much of today's physical AI work assumes a world that is already machine-readable: a clean CAD model, or a synthetic environment built for the purpose. Operating facilities rarely offer that. They are documented across CAD models, drawings, and operational systems with inconsistent structures and identifiers, and even where CAD exists, it may no longer reflect the plant as built. Turning such a facility into a world that software can perceive, simulate, and act within is the precondition for everything that follows, and it is the work Falkor has industrialized. There are two routes into that world. Where usable engineering CAD exists, it is converted, enriched, and regenerated as it changes. Where it is missing or out of date, agents reconstruct a model from drawings and anchor it to as-built captures.

The work, journey by journey

Legible and observable are where it starts. Semantics drive behavior, behavior enables interaction, interaction enables simulation, and simulation enables reasoning. Reasoning is the destination, and it is where the work gets done.

Follow the flow

An agent reads a P&ID and highlights every part that belongs to it, follows a line from one sheet to the next, and puts the process back into the plant, so you see where the job actually is. Where drawings are missing or out of date, an agent infers what connects to what, and which way it runs, from the geometry itself. Where the evidence is thin, it says so: what the available data supports, and how much confidence the result deserves.

Stand at the worksite with the team before anyone commits to the plan

A maintenance planner's questions are physical. Is there clearance to withdraw the valve? Can the crane reach? Which access route still works once the scaffolding is up? Can two crews work the same area at the same time? An agent that can see the plant answers these while the plan is still cheap to change, which is when the answer is worth the most. With interactive physics in the same world, an agent can test whether a lift clears the structure before anyone commits to it. On one maintenance job, an agent checked that there was room for a person to stand and work, found that a short handling zone for the part was clear while a longer one hit the railing, and gave the planner a two-stage option with the open questions listed.

Simulate scenarios

Where a question is genuinely about future behavior, the agent selects and invokes a validated model to compute the answer. The simulator computes; the agent reasons about which model to ask, and about what the result means for the equipment in front of the person. Engineering-grade simulators such as K-Spice and LedaFlow provide validated models of process and flow behavior. Emulators provide faster approximations for bounded questions and broader scenario exploration. NVIDIA physics simulation libraries add spatial interaction, collision, articulation, sensors, and physical behavior within the OpenUSD world.

From every flange on the plant to the short list worth the hours

An inspection engineer starts with more candidates than time, and with the context for the decision spread across systems that were never joined up. An agent applies spatial exposure, process conditions, temperatures, inspection history, and planned work in turn. On one reference facility, that screening narrowed all 4,847 flanges on the plant to 25 candidates for engineering review, with the reasoning behind each one open to challenge. The numbers will differ from one facility to the next. The pattern is what carries over.

Evidence, provenance, and human control

Consequence is asymmetric in this industry. Being wrong about a flange or a lift is not a rounding error, which is why evidence, provenance, and human control are product features here rather than compliance decoration. Industrial agents operate within defined identities, permissions, and workflows. Results distinguish retrieved facts, model outputs, rendered or spatial observations, calculations, and AI inference. Where a drawing shows something but does not locate it, the agent does not invent a position; it leaves the item out of the model. Where two drawings disagree, it keeps the disagreement visible. Every part of a generated model can point back to the drawing it came from. The person sets the objective, sees which sources and models contributed to a conclusion, challenges it, and decides how far it should progress toward action.

A few months of momentum

At Hannover Messe in April, Falkor announced its work with NVIDIA Omniverse libraries. 

Since then, the way the two fit together has changed. The earlier integration model centered on adopting or extending a substantial Kit-based application stack. The new model decomposes key Omniverse capabilities into focused libraries, APIs, microservices, and associated skills, and Falkor has moved its spatial capabilities onto it.

Agents run the pipeline end to end. The scene in the footage above is a dynamic simulation, rendered on Microsoft Azure nodes running NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs.

How it fits together?

^{IMAGE: The Falkor architecture, from the people and agents doing the work at the top to the engineering data underneath. Reasoners give agents a cockpit around specialized engines, including NVIDIA Omniverse libraries, and the graph remains the source of truth.}

The architecture reads from top to bottom, from the people and agents doing the work to the engineering data underneath it.

At the top, a person and an agent share the same session. The agent can be Falkor's Reasoning Engine or another agent harness. The operator watches and instructs it, and the Surface client carries the shared view to the browser. Agents reach the platform through an MCP gateway that handles authentication and data access, and act on the live session through an MCP bridge.

Beneath them sit reasoners. A reasoner is an agent-accessible interface around a specialized engine that combines controls with the observations and context needed to operate it iteratively. The agent performs the reasoning; the reasoner provides its cockpit around the engine. A semantic reasoner works over the source engineering data: structure, identity, and meaning. A scene reasoner works over the composed 3D scene, using Falkor's own Cogs engine for reach and NVIDIA Omniverse for fidelity. An as-built reasoner works over captures of the facility as it stands, starting with laser scans.

The engines are where NVIDIA's libraries do their work, each composed for a specific capability:

• ovstage gives agents a scene data plane to load, query, and change the OpenUSD world, isolating equipment or staging a scenario rather than only looking at it.
• ovrtx renders the scene with equipment identity attached, so an agent knows what it is looking at, pixel by pixel.
• ovphysx adds interactive physics, such as detecting whether a crane collides with the structure during a lift.
• ovstream streams the rendered view into Falkor Surface, in any browser.
• SimReady validation, USD optimization, and CAD ingestion run inside the Falkor CAD Import Engine, as the world is built from engineering CAD.

At the bottom sits the data. Engineering CAD remains authoritative for the engineering design. As-built captures supply measured evidence of the plant as it stands. Falkor's graph is the source of truth for identity, relationships, and engineering semantics. The OpenUSD stage is a derived spatial projection, and asset identity resolves back to the graph.

NVIDIA libraries provide a reusable spatial and physical foundation. Falkor supplies the industrial meaning that makes them useful on a real asset. Libraries are callable capabilities. Skills carry procedural knowledge. Agents perform the composition. The unit of integration moves from the application to the capability.

AI on every side of the table

AI participates on every side of this environment. Engineering agents help teams integrate rapidly evolving technologies and turn industrial capabilities into reusable tools. Falkor's work with NVIDIA Omniverse libraries demonstrates this mode: engineering agents use NVIDIA skills to implement capabilities against early library releases. Upstream agents help prepare the industrial world by inspecting source CAD, resolving identities, proposing classifications, enriching geometry, and validating derived representations. The path from engineering CAD to a simulation-ready spatial world becomes an ongoing reasoning process rather than a one-time file conversion. Within operational workflows, specialist agents retrieve information, calculate, render, measure, simulate, compare, and prepare actions.

Agents are now reliable in code and command-line workflows. Reasoning about 3D space is the next frontier. The step beyond agentic software engineering is agentic spatial engineering.

Frontier models that reason over 3D space, such as OpenAI's GPT-6 Astra, extend what these agents can do: building with Omniverse libraries, composing and validating OpenUSD scenes, and interpreting what an agent sees in a rendered view.

Agentic spatial engineering, in practice

The work follows one line: generate a plant model from evidence, ground it against reality, then simulate and operate on it.

Agents powered by GPT-6 Astra do the spatial reasoning. Underneath them is Falkor's product: the knowledge graph, and reasoners that let agents work directly with the CAD Import Engine.

Where a facility has no up-to-date CAD model, the agent starts from the drawings. It writes the 3D geometry itself, sized from what it reads in P&IDs, isometrics, and arrangement drawings, and joins the sheets into one connected process in OpenUSD. It then anchors that model to as-built captures of the facility, starting with laser scans. Drawings supply the documented equipment and relationships. Captures supply measured spatial evidence. Where documents and capture disagree, the discrepancy is surfaced for engineering review.

At NVIDIA GTC Berlin

Falkor is working with NVIDIA and Microsoft to bring AI and spatial intelligence into a product built for heavy-asset industries, running on Azure AI infrastructure. Because Falkor sources GPT-6 Astra and its other OpenAI models through Microsoft Foundry, operators can apply frontier models to their asset data on a platform they have trusted for many years. The use cases are being developed with input from international energy operators, with an initial focus on inspection and maintenance planning.

In Berlin, Falkor shows an agent connecting geometry to equipment identity and operational context, and using rendered evidence within a governed industrial workflow. The capabilities shown in Berlin are part of the continuous evolution of Falkor Surface and the Falkor Platform, available on Microsoft Marketplace.

To explore what this could mean for your assets, connect with a Falkor expert at falkor.com.