About:

• Parses CIM RDF/XML data to pandas dataframe with 4 columns [ID, KEY, VALUE, INSTANCE_ID] (triplestore like)
• The solution does not care about CIM version nor namespaces
• Input files can be xml or zip files (containing one or mutiple xml files)
• All files are parsed into one and same Pandas DataFrame, thus if you want single file or single data model, you need to filter on INSTANCE_ID column

Documentation:

https://haigutus.github.io/triplets

Upgrading from 0.0.x? See docs/migration_0.0_to_0.1.md.

To get started:

# Core (python_lxml_pandas engine, no extra deps)
pip install triplets

# With pyarrow (enables python_lxml_arrow + cython_pugixml_arrow engines, ~12x faster)
pip install triplets[arrow]

Install extras by feature:

Extra	Enables
arrow	compiled Arrow parser engines (~12x faster parsing)
polars	polars DataFrames (polars.read_rdf, .triplets namespace)
duckdb	DuckDB connections (con.read_rdf, SQL over triplets)
sparql	SPARQL queries (rdflib reference engine)
oxigraph	recommended pip performance path — embedded Rust SPARQL engine (auto-preferred over rdflib)
validation	SHACL validation (pyshacl reference engine)
excel / networkx / visualization	Excel export / graph export / drawing

The embedded qlever SPARQL engine (fastest) ships in no wheel — it is a local source build, see docs/building.md.

import pandas
import triplets

path = "CGMES_v2.4.15_RealGridTestConfiguration_v2.zip"
data = pandas.read_RDF([path])

Result:

You can then query a dataframe of all same type elements and its parameters across all [EQ, SSH, TP, SV etc.] instance files, where parameters are columns and index is object ID-s

data.tableview_by_type("ACLineSegment")

Export:

data.export_to_cimxml(
    rdf_map=schemas.ENTSOE_CGMES_2_4_15_552_ED1,
    export_type=ExportType.XML_PER_INSTANCE_ZIP_PER_XML,
)

Look into examples folders for more

Parser engines

Three parser engines with automatic fallback (fastest available):

Engine	Install	Speed
python_lxml_pandas	pip install triplets	1x baseline, always works
python_lxml_arrow	pip install triplets[arrow]	~1x, better interop
cython_pugixml_arrow	pip install triplets[arrow] (included in wheels)	12x faster

The cython_pugixml_arrow engine is a compiled C++ extension included in published wheels. It requires pyarrow at runtime, so install with triplets[arrow] to enable it.

The cython engine is pre-built in published wheels — no compilation needed.

Polars

import polars
import triplets

data = polars.read_rdf(["grid_EQ.xml", "data.zip"])   # returns polars DataFrame

data.triplets.get_types_count()
data.triplets.tableview_by_type("ACLineSegment")
data.triplets.filter_triplets(KEY="Type", VALUE=".*Generator.*", regex=True)
data.triplets.export_to_csv(export_to_memory=True)
data.triplets.export_to_nquads("/tmp/output.nq")

DuckDB

import duckdb
import triplets

data = duckdb.connect()                              # in-memory
data = duckdb.connect("grid.duckdb")                 # persistent (no re-parsing next session)

data.read_rdf(["grid_EQ.xml", "data.zip"])           # parse via Arrow (zero-copy into DuckDB)
data.get_types_count()                                     # → dict
data.tableview_by_type("ACLineSegment").df()             # → pandas DataFrame
data.tableview_by_type("ACLineSegment").pl()             # → polars DataFrame
data.filter_triplets(KEY="Type", VALUE=".*Sub.*", regex=True).df()
data.filter_triplets_by_type("Terminal").df()
data.references_to("some-uuid").df()
data.export_to_nquads("/tmp/output.nq")

# Direct SQL (full DuckDB SQL on the triplets table)
data.sql("SELECT VALUE, COUNT(*) FROM triplets WHERE KEY = 'Type' GROUP BY VALUE").df()

# The same tools are also on the `.triplets` namespace (parity with pandas/polars)
data.triplets.tableview_by_type("ACLineSegment").df()
data.triplets.get_types_count()

SPARQL queries

SPARQL 1.1 over the loaded data — SELECT → DataFrame, ASK → bool, CONSTRUCT → triplet DataFrame. Works on pandas, polars and DuckDB inputs:

PREFIXES = """
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX cim: <http://iec.ch/TC57/CIM100#>
"""
names = data.sparql.query(PREFIXES + "SELECT ?s ?name WHERE { ?s cim:IdentifiedObject.name ?name }")

Three engines behind one API (auto picks the fastest available):

Engine	Install	Role
qlever	local source build (docs/building.md)	fastest — embedded C++, persistent on-disk index
oxigraph	pip install triplets[oxigraph]	embedded Rust — ~3x faster import, 2–5x faster queries than rdflib
rdflib	pip install triplets[sparql]	pure-Python reference

Details and measured numbers: docs/sparql.md.

SHACL validation

Validate against SHACL shape files; the result is a violations DataFrame (empty = conforms) with the same shape across all engines:

violations = data.shacl.validate("shapes.ttl", rdf_map=schemas.ENTSOE_CGMES_3_0_0_552_ED1)

# slower optional context pass: source file, object type/name,
# shape sh:name/sh:description, schema attribute/class definitions
violations = data.shacl.validate(shapes, context=True)

# SARIF 2.1.0 for GitHub / SonarQube / any SARIF viewer — grouped by default
# (one result per rule with occurrenceCount + sample instances)
violations.shacl.to_sarif(path="report.sarif")

Engines: polars (auto, real profiles in ~2 s) → pandas → pyshacl (reference); duckdb for larger-than-memory data. sh:sparql constraints ride the SPARQL engine above (minutes → milliseconds with oxigraph/qlever). Details: docs/validation.md.

Accessor namespace

pandas and polars DataFrames use df.triplets.*; a DuckDB connection uses con.triplets.*. The same method names are available on both (DuckDB returns relations — add .df() or .pl() when needed):

# pandas / polars
df.triplets.tableview_by_type("ACLineSegment")
df.triplets.export_to_nquads("/tmp/output.nq")

# DuckDB
con.triplets.tableview_by_type("ACLineSegment").df()
con.triplets.get_types_count()

Root-level methods (df.type_tableview(...), con.filter_triplets(...)) still work for backwards compatibility.

CLI tools

cim-spreadsheet -i model.xml -o output.xlsx
cim-diff original.xml modified.xml

Performance (RealGrid, 1.14M rows)

Operation	pandas	polars	DuckDB
Parse (cython engine)	128ms	156ms	283ms
tableview_by_type	72ms	21ms	53ms
filter_triplets_by_type	103ms	9ms	50ms
get_types_count	21ms	11ms	18ms

The old rdf_parser.py functions still work but emit deprecation warnings. See docs/migration_0.0_to_0.1.md for renames and breaking changes.