Skip to main content

Local terminology mode

By default the terminology functions call a remote FHIR terminology server. As an alternative, Pathling can evaluate the same functions against a local terminology store with no network dependency. You import SNOMED CT and FHIR terminology content into the store once, then configure a context for local mode pointing at that store. All seven terminology functions (member_of, translate, subsumes, subsumed_by, display, property_of and designation) work identically in local mode.

Local mode is useful when a terminology server is unavailable, when network access or request volume is a constraint, or when reproducibility across environments matters.

Importing content​

SNOMED CT is imported from an RF2 snapshot release (a .zip archive or an extracted directory). FHIR CodeSystem, ValueSet and ConceptMap resources are imported from a JSON file, a directory of JSON files, or a FHIR NPM package (.tgz). The store is written as Delta tables under a location on any filesystem accessible through the Hadoop FileSystem API, and can be reused across sessions and from cluster deployments. Re-importing a version replaces it atomically.

from pathling import PathlingContext

pc = PathlingContext.create()
pc.import_snomed(
    "/data/SnomedCT_InternationalRF2_PRODUCTION_20250601T120000Z.zip",
    "/data/tx-store",
)
pc.import_fhir_terminology("/data/hl7.terminology.r4-6.5.0.tgz", "/data/tx-store")

RF2 sources must be self-contained​

An RF2 source is imported on its own terms: the concepts it ships are the dictionary that every other file is resolved against. A description, relationship or reference set row referencing a concept the source does not ship has nothing to attach to, so it is dropped. A relationship needs both its source and its destination concept, so one missing destination drops the row.

This is the ordinary shape of a derived or extension package. Such a package declares its dependency on another edition through the Module Dependency reference set and ships only its own modules' components, so most of what it references lives in the edition it extends. Imported alone, it succeeds while carrying almost none of the content you expected. Two published examples: the SNOMED CT International Patient Summary ships no concepts of its own at all, and the SNOMED CT Netherlands Patient Friendly Extension ships two bookkeeping concepts alongside 1,287 active descriptions, of which 7 resolve.

To tell whether this has happened, read the per-file resolution counts in the import log. Every file resolved against the concept dictionary reports one line:

.../sct2_Description_Snapshot-en_NL_20200930.txt: 7 of 1287 active rows resolved against the concept dictionary.

A line is reported for every such file, including the files that resolve completely, so a file whose two figures are equal is never confused with one that was absent. Both figures count active rows only, because rows excluded for being inactive are excluded by design rather than for want of a concept. The concept file itself, the language reference sets and the Module Dependency reference set produce no line, since none of them is resolved against the concept dictionary.

Unresolved rows are reported informationally and never fail the import: importing a package whose references are mostly external is a legitimate thing to do if that is what you intend. The lines are logged at INFO by the importer, so they appear wherever logging for au.csiro.pathling is enabled at that level.

Combining a derived package with its dependency​

To import a derived package's content in full, combine it with the release it declares a dependency on and import the combination as a single source.

Three roles are single-valued, so their files must be concatenated into one file each, keeping only the first file's header row:

  • the concept file (sct2_Concept_Snapshot_*)
  • the relationship file (sct2_Relationship_Snapshot_*)
  • the Module Dependency reference set (der2_ssRefset_ModuleDependencySnapshot_*)

Every other role is multi-valued, so those files are left as they are, each keeping its own header, in the directory layout the import expects:

  • descriptions and text definitions (sct2_Description_*, sct2_TextDefinition_*)
  • language reference sets (der2_cRefset_Language*)
  • all other reference sets (der2_*Refset*)
INT=/data/SnomedCT_InternationalRF2_PRODUCTION_20250601T120000Z/Snapshot
EXT=/data/SnomedCT_ExtensionRF2_PRODUCTION_20250930T120000Z/Snapshot
OUT=/data/merged/Snapshot

mkdir -p "$OUT/Terminology" "$OUT/Refset/Language" "$OUT/Refset/Content" \
         "$OUT/Refset/Metadata"

# Single-valued roles: concatenate, keeping only the first file's header row.
concat() {
  cat "$1" > "$3"
  tail -n +2 "$2" >> "$3"
}
concat "$INT"/Terminology/sct2_Concept_Snapshot_*.txt \
       "$EXT"/Terminology/sct2_Concept_Snapshot_*.txt \
       "$OUT/Terminology/sct2_Concept_Snapshot_MERGED.txt"
concat "$INT"/Terminology/sct2_Relationship_Snapshot_*.txt \
       "$EXT"/Terminology/sct2_Relationship_Snapshot_*.txt \
       "$OUT/Terminology/sct2_Relationship_Snapshot_MERGED.txt"
concat "$INT"/Refset/Metadata/der2_ssRefset_ModuleDependencySnapshot_*.txt \
       "$EXT"/Refset/Metadata/der2_ssRefset_ModuleDependencySnapshot_*.txt \
       "$OUT/Refset/Metadata/der2_ssRefset_ModuleDependencySnapshot_MERGED.txt"

# Multi-valued roles: copy every file as it is. The two releases name their files
# by edition and date, so nothing is overwritten.
cp "$INT"/Terminology/sct2_Description_*.txt \
   "$INT"/Terminology/sct2_TextDefinition_*.txt \
   "$EXT"/Terminology/sct2_Description_*.txt "$OUT/Terminology/"
cp "$INT"/Refset/Language/*.txt "$EXT"/Refset/Language/*.txt "$OUT/Refset/Language/"
cp "$INT"/Refset/Content/*.txt "$EXT"/Refset/Content/*.txt "$OUT/Refset/Content/"

pathling import-snomed /data/merged /data/tx-store

Adjust the copies for the roles a given package actually ships; an extension without text definitions, for instance, contributes none.

Do not simply extract both releases side by side into one directory. The import rejects a source in which more than one file fills a single-valued role, naming the role and every candidate path, because it cannot tell which tree's content you meant and would otherwise proceed against one and silently ignore the other.

Combine a package only with the release it declares a dependency on. An extension ships only its own modules' components, so a package and its dependency do not overlap. Two overlapping editions do: the same concept code would arrive twice, be given two internal identifiers, and fan out every join built on it. Nothing detects that, so it is yours to avoid.

Reducing the memory the hierarchy takes at query time​

The largest structure a local store loads into memory is the hierarchy index, which holds the transitive closure of the is-a graph as compressed bitmaps addressed by an internal identifier per concept. By default those identifiers are assigned in concept code order, and a code's numeric value bears no relation to the concept's place in the hierarchy, so a concept's descendants scatter across the whole identifier range and compress poorly.

The pre-order setting instead assigns identifiers by a depth-first traversal of the is-a hierarchy, so each subtree occupies a near-contiguous interval. Measured over a full SNOMED CT UK edition of 1,115,237 concepts, this reduces the hierarchy index from 738 MB to 536 MB of retained heap, a saving of 27%.

The trade-off is identifier stability. Under the default ordering a concept keeps its identifier across re-imports of any release that contains it, and identifiers change only where codes are added or removed. Under the pre-order, a change anywhere in the shape of the hierarchy shifts the identifiers of everything that follows it, so identifiers vary much more between releases. Identifiers are internal to a store and never appear in query results, so this affects nothing a user can observe directly; it matters only if you compare or reuse the internal identifiers of two separately imported stores. Repeated imports of the same release remain reproducible under both orderings, and all seven terminology functions return identical results either way.

pc.import_snomed(
    "/data/rf2.zip",
    "/data/tx-store",
    dense_id_order="pre-order",
)

Large CodeSystems​

CodeSystems are imported with bounded memory regardless of their size. The import streams each CodeSystem from the source, transcodes it to temporary files on the driver, and loads it with Spark, so peak memory does not grow with the number of concepts and a single CodeSystem may exceed the 2 GB limit on a single in-memory object (for example, the OMOP vocabulary package's multi-gigabyte CodeSystem). This applies equally to a bare JSON file, a directory, and a .tgz package.

Peak memory is bounded, but the largest vocabularies still need more than the default 1 GB driver heap to hold the working set of the Spark joins that build the store; the OMOP vocabulary, for example, imports comfortably with a 4 GB heap. See driver memory for large imports in the CLI guide for how to raise it, which applies equally to imports run through the library.

During a long import, a running count of parsed concepts is logged at a fixed interval alongside stage-transition messages, so progress is visible rather than appearing to hang. The CLI surfaces these messages in --verbose mode.

Hierarchies from parent and child properties​

Many flat CodeSystems express their hierarchy through parent (or child) concept properties rather than nested concepts. The import derives is-a edges from code-valued parent and child properties, recognised by the standard parent/child property codes or a property declaration carrying the standard concept-properties URI, in addition to concept nesting. Edges from both sources are combined, so subsumption and descendant-based membership queries work over property-based hierarchies just as they do over nested ones. A parent or child reference to a code absent from the CodeSystem is skipped with a warning, and duplicate concept codes resolve to their first occurrence with a warning.

Large ConceptMaps​

ConceptMaps are imported with bounded memory regardless of their size, in the same way as CodeSystems: each is streamed from the source to temporary files on the driver and loaded with Spark, so the driver never holds a map's mappings in memory. Each target of each source element is stored as a row of the store's concept_mapping table, and a running count of parsed mappings is logged during a long import.

At query time a map's mappings are read the first time translate uses it, and only those of the version that answers, so maps that are never used cost nothing. The mappings are then held in memory on each executor for the life of the session, in a compact form that stores each distinct code once as UTF-8 bytes and each mapping as a few integers. In measurements of maps of up to 20 million mappings, where nearly every code was distinct, this came to about 53 bytes per mapping: 267 MB for 5 million mappings and 1 GB for 20 million. Executor memory must accommodate the maps a query uses.

Where the store holds several versions of a ConceptMap, translate uses the latest, chosen by the same version ordering as for ValueSets. The versions are never combined.

Only the parts of a ConceptMap that Pathling reads are stored: the source and target systems and versions of each group, the code and display of each element, and the code, display and equivalence of each of its targets. For translate, a target without an equivalence is treated as relatedto, and an element without a code is skipped. A map is also read in full, as rows of source and target codes; content those rows cannot carry (a group without a source, an element without a code, or a target with dependsOn or product, without an equivalence, or without a code when it is not unmatched) does not stop the import or translate, but reading the map in full fails, naming the first such fault in the map, which the import records in the fault column of the manifest. An unrecognised equivalence fails the import, naming the map, before any of the map is written.

Bundles and non-CodeSystem resources​

A Bundle is streamed entry by entry, so CodeSystems and ConceptMaps inside a Bundle are imported with the same bounded memory as standalone ones. Of each entry, only its type, canonical URL and version are held in memory, never its content. The entries are validated before anything is written: each CodeSystem, ValueSet or ConceptMap must carry a canonical URL, and an entry holding any other kind of resource is skipped.

ValueSets are stored whole, so a single ValueSet must fit in memory; one larger than 1 GB, standalone or in a Bundle, fails with an actionable error naming the resource rather than an opaque memory error.

Sources of many resources​

An import reads every resource of the source into temporary files on the driver before it writes anything to the store, then writes each table of the store once for the whole source, rather than once per resource. A package of thousands of small resources, such as hl7.fhir.r4.core with its 2,458 CodeSystems, ValueSets and ConceptMaps, is imported with one write per table rather than thousands.

Where a source holds the same resource (the same resource type, canonical URL and version) more than once, the last occurrence is imported and the earlier ones are skipped with a warning.

Stores written by earlier versions​

Format version 2 changed how ConceptMaps are stored, from one JSON document per map to one row per mapping, which is what makes large maps possible. A store written by Pathling 9.9.0 or earlier is migrated to format version 2 in place when a Pathling context is created with it in local mode. Each ConceptMap is read from its stored JSON and written as mapping rows, exactly as importing it would write them, and its manifest row is updated to format version 2 and given the fault an import would record, keeping its source and imported_at. The table that held the JSON documents is then removed. Code systems and value sets did not change layout, so they are left as they are and their manifest rows keep format version 1. A store with no ConceptMaps in the earlier layout has nothing to migrate, so it may be read-only.

Migration needs write access to the store. If it cannot write, context creation fails with an error naming the store; nothing is recorded as migrated until the mappings are written, so creating the context again with write access completes the migration. Re-importing the FHIR terminology content into a new store also works.

A SNOMED CT edition imported by Pathling 9.9.0 or earlier did not record the targets of simple map reference sets, which cannot be recovered from the store, so translation through those reference sets as implicit concept maps finds nothing. Context creation logs a warning naming each such edition until it is re-imported. Everything else about the edition works unchanged.

A store written or migrated by this version of Pathling does not open with Pathling 9.9.0, which reports the format version as too new.

Recovering from a failed import​

A resource that cannot be read (for example, a CodeSystem that is truncated or corrupt, or a ConceptMap that carries an unrecognised equivalence) fails the import with an error naming it. Because the whole source is read before anything is written, the store is left unchanged.

If an import fails while it is writing the store, it reports that the store may hold partial versions of the resources being imported and advises re-running the import. The manifest is written last, so none of them is recorded as imported. Because content is keyed by resource version, re-running with a corrected source fully replaces the partial versions and repairs the store.

Provenance and verification​

Every import records the provenance of what it loaded in the store's manifest table, a Delta table under the store root. There is one row per imported CodeSystem, ValueSet or ConceptMap, keyed by canonical URL and version; re-importing replaces the row, so a row always describes the most recent import of that content. Every row written by a single import carries the same provenance values.

ColumnTypeNullMeaning
store_format_versionintnoThe store layout version the row was written under: 2, or 1 for a code system or value set row written by Pathling 9.9.0 and earlier.
entry_typestringnocode_system, value_set or concept_map.
canonical_urlstringnoThe canonical URL of the imported resource.
versionstringyesIts version, where the resource declares one.
sourcestringyesThe source path, as passed to the import.
imported_attimestampyesWhen the import wrote the row.
source_sha256stringyesThe SHA-256 of the source file's bytes, as 64 lowercase hexadecimal characters. Null for a directory source, which has no single set of bytes, and for rows written before provenance was recorded.
package_namestringyesThe name from the package's package.json. Null unless the source is a FHIR NPM package that declares one.
package_versionstringyesThe version from the package's package.json, with the same nullability as package_name.
package_verificationstringyesverified, unverified or skipped. Null when the source was not a package.
package_registrystringyesThe registry that vouched for the bytes, with any trailing slash removed. Set only when package_verification is verified.
faultstringyesWhy a ConceptMap cannot be read in full as rows of source and target codes, naming the first such fault in the map. Null where it can, and for code systems and value sets.

The store format version is unchanged by the last six columns, so a store built before they were recorded opens, answers queries and accepts new imports without migration, its existing rows reading back as null.

The SHA-256 is taken from the bytes the import already reads, so it adds no further pass over an archive, and it covers the whole file: it agrees with shasum -a 256 run on the same file. Where a publisher gives a SHA-256 out of band, for example in an NCTS syndication feed entry, source_sha256 can be compared with it directly. This is the only fingerprint available for an RF2 release or an unpublished package, neither of which has a registry checksum.

What is checked against the registry​

FHIR package registries implement the npm registry protocol: GET {registry}/{name} returns a listing of every published version, and a version may carry dist.shasum, the SHA-1 of the tarball the registry distributes. When the source of a FHIR terminology import is a package (.tgz or .tar.gz), the import reads the name and version from the package's package.json, fetches that listing, and compares the dist.shasum published for that version, case-insensitively, with the SHA-1 of the tarball in hand.

The check applies to packages only. An RF2 release, a directory of JSON files and a single JSON file are not distributed through a package registry, so no registry is consulted for them, no verification warning is raised, and package_verification and package_registry are left null. That holds for a directory even when it contains a package.json. The check is made at import time, against the bytes being read; nothing is re-checked when the store is later opened for querying.

The outcome recorded in package_verification is one of:

  • verified: the registry published a shasum for the version and it equals the tarball's SHA-1. package_registry names the registry consulted.
  • unverified: the source was a package, but no comparison could be made. The import warns with the reason and proceeds, and package_registry is null. The reasons are: the package has no package.json, or it declares no name or version, in which case no registry is contacted at all; the registry could not be reached, or did not answer within the timeout; it answered with an error status, or with something that is not a version listing; it does not list the package, or does not list the version; or it lists the version without a shasum. A registry that cannot answer is not evidence that the bytes are wrong, so none of these fails the import.
  • skipped: the source was a package and verification was turned off. No registry is contacted, and the package name and version are still recorded.
  • null: the source was not a package.

A published checksum that differs is the only outcome that fails the import. The failure comes before any table or manifest row is written, so a store that did not exist beforehand is not created, and an existing store is left as it was. The message names the package, the version, the registry, the expected SHA-1 and the SHA-1 of the source in hand, and how to import the package anyway:

The package hl7.terminology.r4 6.5.0 does not match the registry checksum published by https://packages.fhir.org: expected SHA-1 8a7a096866b9b6e96e288ce8d72bf99aa31714a3 but the source has SHA-1 3b1f0f16d6f0e14a29b0a4ba3a0b0e9d7c5a1f42. The source may be corrupt or tampered with; import it anyway by turning off the verifyPackage option.

The library logs the outcome: a verified package at INFO, naming the package, its version, the registry and the SHA-1; an unverified one at WARN, naming the reason. Both appear wherever logging for au.csiro.pathling is enabled at that level. The CLI runs the library at WARN, so --verbose shows the reason for an unverified package; the verified outcome is reported on the completion line instead.

Importing offline, and choosing a registry​

Verification is on by default and can be turned off for a single import, which is how a package is imported with no network access, or a package that was never published to a registry. The registry can also be pointed elsewhere, for instance at Simplifier for a package that packages.fhir.org does not mirror. The default registry is https://packages.fhir.org, and a trailing slash on a supplied URL makes no difference. Both settings are accepted for a source that is not a package, where they have no effect.

pc.import_fhir_terminology(
    "/data/my-own-package.tgz", "/data/tx-store", verify_package=False
)
pc.import_fhir_terminology(
    "/data/hl7.terminology.r4-6.5.0.tgz",
    "/data/tx-store",
    package_registry="https://packages.simplifier.net",
)

In Java, both are set on FhirImportOptions, passed to PathlingContext.importFhirTerminology.

Comparing two stores​

Two environments that imported the same content can be compared by reading both manifests and taking the rows that appear in one and not in the other. An empty result means the two imported byte-identical sources.

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()
columns = [
    "canonical_url",
    "version",
    "source_sha256",
    "package_verification",
    "package_registry",
]
a = spark.read.format("delta").load("/env-a/tx-store/manifest").select(columns)
b = spark.read.format("delta").load("/env-b/tx-store/manifest").select(columns)
a.exceptAll(b).union(b.exceptAll(a)).show(truncate=False)

Querying in local mode​

Create a context configured for local mode by setting the terminology mode to local and pointing at the store. The terminology functions then evaluate against the store.

from pathling import PathlingContext
from pathling.functions import to_snomed_coding
from pathling.udfs import member_of
from pyspark.sql import functions as F

pc = PathlingContext.create(
    terminology_mode="local",
    terminology_storage_path="/data/tx-store",
)

result = df.select(
    "id",
    member_of(
        to_snomed_coding(F.col("code")),
        "http://snomed.info/sct?fhir_vs=ecl/<< 73211009 |Diabetes mellitus|",
    ).alias("is_diabetes"),
)

The following configuration parameters control local mode:

  • terminology_mode (terminology.mode): server (the default) or local.
  • terminology_storage_path (terminology.local.storagePath): the store location, required in local mode.
  • default_snomed_edition (terminology.local.defaultSnomedEdition): the SNOMED CT module identifier used to disambiguate an unversioned SNOMED reference when the store holds multiple editions.
  • expansion_cache_size (terminology.local.expansionCacheSize): the maximum number of value set expansions cached per executor.
  • dialect_aliases (terminology.local.dialectAliases): additional dialect tags recognised when a display is requested in a particular language. See dialects.

Dialects​

Within SNOMED CT, which of a concept's synonyms is its preferred term is not a property of the concept but of a language reference set. Two reference sets of the same language routinely disagree: the International edition ships both GB English and US English, and 32849002 is "Oesophageal structure" in the first and "Esophageal structure" in the second. A dialect is the caller-facing name for one of those reference sets.

Naming a dialect​

A dialect may be named in any of three ways, and all three are interchangeable:

FormExampleNotes
A recognised tagen-GBMatched without regard to case. The recognised tags are below.
An extension tagen-x-sctlang-90000000-00005080-04The form Pathling reports as the language of a preferred designation, so a language reported on the way out can be requested on the way in.
A reference set identifier900000000000508004Accepted by the import option only, not by a query-time language request.

The following tags are recognised out of the box. They cover the language reference sets defined in the SNOMED CT International edition; a reference set defined inside a national extension is reached through an alias or through the extension tag form.

TagLanguage reference set
en-GB900000000000508004
en-US900000000000509007
en-AU32570271000036106
es448879004
fr722131000
de722130004
ja722129009
zh722128001

A tag naming no reference set - a bare en, or a region nothing covers - expresses no preference rather than an error, and the stored display answers.

Requesting a term in a dialect​

The accept_language context parameter, and the parameter of the same name on display() and property_of(), select by dialect:

pc = PathlingContext.create(
    spark_conf={"spark.driver.memory": "8g"},
    terminology_mode="local",
    terminology_storage_path="/data/tx-store",
)

# "Oesophageal structure", the term GB English prefers.
british = property_of(coding, "display", accept_language="en-GB")

# "Esophageal structure", the term US English prefers.
american = property_of(coding, "display", accept_language="en-US")

A weighted list is read as RFC 9110 describes, and each dialect is tried in descending order of weight until one yields a term. With accept_language="en-NZ;q=0.9,en-GB;q=0.5" against a store holding no New Zealand reference set, the GB English term answers. A tag given zero weight is never used, and a lone * expresses no preference.

The default dialect of a store​

Every concept in the store carries one stored display, which is what a request naming no dialect - or naming one the store cannot serve - receives. That display is the preferred synonym of a single dialect, chosen when the release is imported:

  1. The dialect named by the default_dialect import option, if one is given.
  2. The sole language reference set, where the release holds only one.
  3. US English, where the release is the SNOMED CT International edition.

A release that holds several language reference sets and is not the International edition fails the import, listing every candidate by identifier and by the name the release itself gives it, so that one can be named:

The release holds 3 language reference sets and none of them is a clear default. Name one with the defaultDialect import option:
  900000000000508004  Great Britain English language reference set
  999000691000001104  National Health Service realm language reference set (pharmacy part)
  999001261000000100  NHS realm language reference set (clinical part)

No SNOMED CT release declares which of its language reference sets is the default, so where the release is genuinely ambiguous the choice is the operator's rather than a guess. Nothing is written to the store when the import fails this way.

pc.import_snomed("/data/rf2.zip", "/data/tx-store", default_dialect="en-GB")

Where the chosen dialect marks no preferred synonym for a concept, its display falls to the preferred synonym of the lowest-numbered other language reference set, then to its fully specified name, and finally to its own code.

Registering additional dialects​

A deployment can register its own dialect tags, which is how a reference set defined inside a national extension is reached by a familiar name. An entry for a tag that is already recognised replaces the built-in mapping, so a built-in entry can be corrected.

pc = PathlingContext.create(
    spark_conf={"spark.driver.memory": "8g"},
    terminology_mode="local",
    terminology_storage_path="/data/tx-store",
    dialect_aliases={"en-NZ": "271000210107"},
)

Aliases affect the selection of a display only. The designations of a concept are returned regardless of the language requested, and a synonym preferred within a language reference set is labelled with the extension tag form rather than an alias - see multi-language support. Aliases are not consulted by an import, which receives no service configuration; a reference set outside the recognised tags is named there by its identifier.

The R binding can carry at most ten aliases, a limit of how sparklyr passes a map to the JVM. The Java, Python and command line surfaces have no such limit.

Code systems that are not SNOMED CT​

A FHIR CodeSystem carries plain BCP-47 designation languages with no reference set to resolve, so there a language request is matched against the designation languages directly. A designation whose tag matches the request exactly is preferred over one matching only on the primary subtag, and within each of those one whose use is display is preferred. An extension tag has no meaning outside SNOMED CT, so it falls back to its plain language subtag.

Value set and concept map expressions​

Local member_of resolves three kinds of value set reference, and local translate two kinds of concept map reference. A reference that resolves to content the store does not hold yields the same "unknown content" results as remote mode rather than an error: a null membership from member_of, and no translations from translate.

A value set is evaluated over a single code system version, so a code without a system, such as a string passed to member_of or to the FHIRPath memberOf function, takes the system of that code system.

ReferenceForm
An imported FHIR ValueSetits canonical URL, optionally with |version
Every SNOMED CT concepthttp://snomed.info/sct?fhir_vs
A SNOMED CT reference sethttp://snomed.info/sct?fhir_vs=refset/[refsetId]
A SNOMED CT subtype hierarchyhttp://snomed.info/sct?fhir_vs=isa/[conceptId]
A SNOMED CT ECL expressionhttp://snomed.info/sct?fhir_vs=ecl/[expression]
A VCL expressionhttp://fhir.org/VCL?v1=[expression]
An imported FHIR ConceptMapits canonical URL
A SNOMED CT implicit concept maphttp://snomed.info/sct?fhir_cm=[refsetId]

The isa/ form is the subtype hierarchy including the named concept itself, so it is equivalent to ecl/<<[conceptId]. Any other fhir_vs value is treated as unknown content.

Every SNOMED form, including fhir_cm, is also accepted on an edition URI, with or without a version, and evaluates against that edition rather than the store default:

Base URIEvaluates against
http://snomed.info/sct/32506021000036107/version/20250630that version
http://snomed.info/sct/32506021000036107the latest stored version of that edition

The same applies to a SNOMED CT version given in Coding.version or in a VCL system, such as (http://snomed.info/sct|http://snomed.info/sct/32506021000036107). An edition URI in the http://snomed.info/xsct namespace, which the SNOMED CT URI Standard reserves for unpublished content, selects only from versions imported under that namespace, and an http://snomed.info/sct edition URI only from the others.

A SNOMED CT implicit concept map is either one of the four association reference sets or a simple map reference set, which are the two kinds THO defines as implicit concept maps. Each translation through an association reference set carries the equivalence defined for that reference set, in either direction:

Reference setIdentifierEquivalence
POSSIBLY EQUIVALENT TO900000000000523009inexact
REPLACED BY900000000000526001equivalent
SAME AS900000000000527005equal
ALTERNATIVE900000000000530003inexact

A simple map reference set is any reference set that descends from 900000000000496009 |Simple map type reference set| in the release, or one of the three simple maps in the table below, which Ontoserver also supports. It maps SNOMED CT concepts to codes in another code system. RF2 does not say which code system a map's targets belong to, so the target system is taken from the same table Ontoserver uses, along with the equivalence it reports, in either direction:

Simple mapIdentifierTarget systemEquivalence
ICD-O446608001http://hl7.org/fhir/sid/icd-o-3inexact
CTV3900000000000497000http://read.info/ctv3equivalent
ARTG11000168105https://www.tga.gov.au/australian-register-therapeutic-goodsinexact

CTV3 is accepted even though current releases no longer place it under the simple map type. A translation through any other simple map is inexact, and its target is returned as a code with no system. In the reverse direction, the coding being translated must be in the map's target system where the table names one, and is matched on its code alone where it does not. The extended and complex maps, such as the ICD-10 map, are not simple maps and are not translated.

A concept with more than one target, such as an ambiguous concept that is possibly equivalent to several others, translates to all of them. Any other fhir_cm reference set, including other association reference sets such as WAS A, is treated as unknown content.

Only the four association reference sets can also be read as a table with concept_map (pathling_concept_map in R, conceptMap in Java and Scala), as described in Terminology in SQL. A simple map reference set can be used with translate but not read as a table, and is rejected with an error that says so.

A store imported by Pathling 9.9 holds no simple map targets, so translation through a simple map finds nothing until the release is imported again.

The expression carried by an ecl/ or v1= URL is percent-decoded before it is parsed, so it must be percent-encoded when the URL is built. For ECL, to_ecl_value_set does this (tx_to_ecl_value_set in R, toEclValueSet in Java and Scala); there is no equivalent helper for VCL, so encode it yourself.

Both expression languages are evaluated by the same engine: an ECL expression is translated into the VCL model, and the result is evaluated against the store's indexes. Members are restricted to active concepts, matching the behaviour of a terminology server on an implicit value set, unless the expression explicitly asks for inactive concepts with an inactive = true filter.

An imported FHIR ValueSet's compose is translated into the same model, so its filters behave as the matching VCL filters below. The in and not-in operators take a comma-separated list of codes, which selects (or, for not-in, excludes) the listed concepts. Over SNOMED CT, a single code is instead read as a reference set: concept in X selects the members of reference set X rather than the concept X, as the SNOMED CT filters in FHIR define it, and not-in selects every active concept outside those members. For example, concept in 929360061000036106 selects the members of that reference set, while concept in 73211009,46635009 selects those two concepts.

SNOMED CT Expression Constraint Language​

The grammar recognises the whole of ECL v2, and the translator maps the subset below. A construct outside the subset is rejected with an error naming it, so a query is never quietly answered with the wrong members. A malformed expression raises Invalid ECL expression at position [n]: [reason].

ConstructExample
Concept reference, with an optional term73211009 |Diabetes mellitus|
Wildcard*
Descendant or self, descendant<< 73211009, < 73211009
Child of<! 73211009
Ancestor or self, ancestor>> 73211009, > 73211009
Parent of>! 73211009
Reference set membership^ 447562003
Conjunction, disjunction, exclusion<< 73211009 MINUS << 46635009
Grouping(< 19829001) OR (< 301867009)
Attribute refinement< 404684003 : 363698007 = << 39057004
Attribute negation< 404684003 : 363698007 != << 39057004
Descendant attribute types< 404684003 : << 47429007 = << 39057004
Attribute sets, with OR binding loosest< 404684003 : 363698007 = 1, 246075003 = 2 OR 47429007 = 3
Nested attribute sets< 404684003 : (363698007 = 1 OR 246075003 = 2)
Dotted attribute navigation< 19829001 . 116676008

A hierarchy operator applies to a concept reference only. An attribute name may carry < or <<, which broadens the constraint to descendant attribute types.

The viral infection expression under value set membership is within the subset: its two attributes are separated by a comma rather than grouped into a role group.

Each rejected construct raises Unsupported ECL construct: [construct], where the construct is one of:

ConstructECL
grouped attributes ({ ... })< 404684003 : { 363698007 = 1 }
attribute cardinality ([min..max])< 404684003 : [1..2] 363698007 = 1
reverse attribute flag (R)< 404684003 : R 363698007 = 1
wildcard attribute name< 404684003 : * = 1
hierarchy operator other than < or << on an attribute name< 404684003 : >> 363698007 = 1
hierarchy operator on an attribute name< 19829001 . << 116676008
child-or-self operator (<<!)<<! 73211009
parent-or-self operator (>>!)>>! 73211009
hierarchy operator applied to a wildcard, reference set, or compound expression<< (73211009 OR 46635009)
reference set membership over a wildcard or expression^ *
concrete value (#number or "string")< 404684003 : 1142135004 = #20
term, definition status, or member filter ({{ ... }})<< 73211009 {{ term = "diabetes" }}
history supplement ({{ + ... }})<< 73211009 {{ + HISTORY }}

FHIR ValueSet Compose Language​

The VCL grammar is transcribed from the FHIR IG Guidance specification, and the whole of VCL v1 parses. A malformed expression raises Invalid VCL expression at position [n]: [reason].

An expression must name its code system with a (systemUri) scope, because a value set is evaluated over one code system version. The first scope in the expression selects that version, optionally by |version; a subexpression scoped to any other system contributes no members. An expression carrying no scope at all is treated as unknown content.

ConstructExampleMeaning
System scope(http://loinc.org|2.74)4548-4a code in a version of a system
Code4548-4one concept
Wildcard*every concept
ConjunctionA,Bin both
DisjunctionA;Bin either
ExclusionA - Bin A but not B
Grouping(http://loinc.org)(41995-2;4548-4)a scope or operator over a group
Equalityconcept = 73211009exactly that concept
Is-a, is-not-aconcept << X, concept ~<< Xthat subtype hierarchy, or every active concept outside it
Descendant, childconcept < X, concept <! Xstrict descendants, direct children
Generalisesconcept >> Xancestors of the concept, and itself
Descendant leavesconcept !!< Xdescendants that have no children
Regular expressioncode / "A.*"a property value fully matching the pattern
In, not in a code listconcept ^ {A,B}, concept ~^ {A,B}membership in an enumerated list (SNOMED CT: see below)
In a filter list363698007 ^ {concept << X}membership in a nested constraint
Existsingredient ? truewhether the property is present
NavigationX.parent, {A,B}.parent, *.116676008the values a property takes

The properties a filter may name are:

  • concept, which applies the hierarchy operators to the subsumption graph. For a code system that is not SNOMED CT that graph comes from the CodeSystem's nesting and its parent and child properties, as described under hierarchies from parent and child properties. Over SNOMED CT, concept ^ {A,B} names reference sets: it selects the members of reference set A or B, as ^ A OR ^ B does in ECL, and ~^ selects every active concept outside them.
  • parent and child, which take one edge rather than the closure, so parent = X selects the children of X and child = X selects its parents.
  • inactive, moduleId, sufficientlyDefined and effectiveTime, which are the SNOMED CT concept metadata held in the store.
  • Any property declared by an imported FHIR CodeSystem, over which =, /, ^, ~^ and ? behave as described above and any other operator falls back to an exact match.
  • Any other property, which is read as a SNOMED CT attribute type, so 363698007 = 39057004 selects the concepts having that attribute with that value.

Three limitations are worth knowing:

  • A value set inclusion, whether ^http://example.org/ValueSet/x at the top level or property ^ http://example.org/ValueSet/x as a filter value, parses but contributes no members. Enumerate the codes or nest a filter list instead.
  • An attribute filter matches the value named, and does not apply its operator to that value: 363698007 << 39057004 is the same as 363698007 = 39057004. To match the value's subtree, nest the constraint as 363698007 ^ {concept << 39057004}, or write the whole thing in ECL, where 363698007 = << 39057004 does expand the value.
  • The exclusion operator must be surrounded by whitespace, because a hyphen is a valid character within a code: A - B excludes B from A, while A-B is the single code A-B. For the same reason a code containing a full stop must be quoted, otherwise it parses as property navigation: "B.123", not B.123.

This example selects diabetes mellitus and its subtypes, less type 1 diabetes and its subtypes, over the store's default SNOMED CT edition:

from urllib.parse import quote

VCL = "(http://snomed.info/sct)concept << 73211009 - concept << 46635009"
value_set = "http://fhir.org/VCL?v1=" + quote(VCL, safe="")

result = df.select("id", member_of(to_snomed_coding(F.col("code")), value_set))