Compare Workflow#

PANORAMA provides two comparison subcommands that share a common graph-based strategy:

Subcommand

Entry point

What it compares

compare_spots

panorama.compare.spots.launch()

Genomic islands (RGP spots) across pangenomes

compare_systems

panorama.compare.systems.launch()

Detected biological systems across pangenomes

Both rely on a Gene Family Relatedness Ratio (GFRR) metric computed in panorama.compare.utils and follow the same three-phase structure: build a graph → score edges → cluster into conserved groups.

Shared infrastructure (panorama.compare.utils)#

GFRR — the similarity metric#

The GFRR between two objects (spots or systems) from different pangenomes is the fraction of their gene families that are considered equivalent (via Akin links established by panorama align or panorama cluster):

GFRR(A, B) = |shared Akin families| / aggregation(|families in A|, |families in B|)

The aggregation function is controlled by --gfrr_metrics (min_gfrr, max_gfrr, mean_gfrr). --gfrr_cutoff sets the threshold below which an edge is dropped.

panorama.compare.utils.compute_gfrr() computes the score for a single pair; it is called from both comparison modules.

Clustering#

panorama.compare.utils.cluster_on_gfrr() applies connected-component clustering on the filtered graph. Each connected component becomes one ConservedSpots or ClusterSystems group.

Spots comparison (panorama.compare.spots)#

1. Validate and load#

panorama.compare.spots.check_compare_spots_args() validates arguments and returns the need_info dict. Pangenomes are loaded in parallel by panorama.format.read_binaries.load_pangenomes() with spots and RGPs enabled.

2. Build per-pangenome spot graphs#

panorama.compare.spots.create_pangenome_spots_graph() builds a networkx graph for each pangenome where nodes are Spot objects and edges connect spots that share gene families within that pangenome.

3. Build the cross-pangenome graph#

panorama.compare.spots.create_spots_graph() merges the per-pangenome graphs and adds cross-pangenome edges by calling panorama.compare.spots.compute_gfrr_edges(), which iterates over every pair of pangenomes and scores each spot–spot combination using GFRR.

Optional: overlay systems#

If --systems is passed, panorama.compare.spots.add_systems_info() annotates each spot node with the systems detected within it, and panorama.compare.spots.create_pangenome_system_graph() / panorama.compare.spots.create_systems_graph() build a parallel systems layer linked to the spot graph via panorama.compare.spots.graph_systems_link_with_conserved_spots().

4. Cluster and write outputs#

panorama.compare.utils.cluster_on_gfrr() groups connected spots into ConservedSpots objects. panorama.compare.spots.write_conserved_spots() writes the TSV and optional graph outputs.

Data-flow summary#

load_pangenomes()
  │
  ├─ create_pangenome_spots_graph()   per-pangenome graph
  ├─ create_spots_graph()             merge + GFRR cross-pangenome edges
  │    └─ compute_gfrr_edges()        score every spot pair
  │
  ├─ [optional] add_systems_info()    overlay systems on spots
  │
  ├─ cluster_on_gfrr()                connected components → ConservedSpots
  └─ write_conserved_spots()          TSV + graph outputs

Systems comparison (panorama.compare.systems)#

1. Validate and load#

panorama.compare.systems.check_compare_systems_args() validates that the number of --sources matches --models and returns the need_info dict. Pangenomes are loaded with systems and family metadata enabled.

2. Build per-pangenome system graphs#

panorama.compare.systems.create_pangenome_system_graph() builds a graph for each pangenome where nodes are System objects and edges connect systems that share gene families within that pangenome. panorama.compare.systems.add_system_metadata_to_graph() annotates nodes with system metadata for downstream output.

3. Build the cross-pangenome graph#

panorama.compare.systems.create_systems_graph() merges per-pangenome graphs and panorama.compare.systems.compute_gfrr_edges() scores every system–system pair across pangenomes using GFRR.

4. Cluster and write outputs#

panorama.compare.utils.cluster_on_gfrr() groups connected systems into ClusterSystems objects. panorama.compare.systems.write_conserved_systems() writes TSV outputs.

Optionally, panorama.compare.systems.generate_heatmap() and panorama.compare.systems.create_pangenome_systems_heatmaps() produce Bokeh HTML heatmaps of system distributions across pangenomes.

Data-flow summary#

load_pangenomes()
  │
  ├─ create_pangenome_system_graph()  per-pangenome graph
  ├─ create_systems_graph()           merge + GFRR cross-pangenome edges
  │    └─ compute_gfrr_edges()        score every system pair
  │
  ├─ cluster_on_gfrr()                connected components → ClusterSystems
  ├─ write_conserved_systems()        TSV outputs
  └─ [optional] generate_heatmap()    Bokeh HTML visualisation

Parallel execution#

Both workflows wrap their per-pangenome graph construction in a ThreadPoolExecutor and use a shared multiprocessing.Manager().Lock() (via panorama.utils.init_lock()) to serialise concurrent HDF5 reads. The number of workers is controlled by --cpus.

Note

Alignment or clustering (panorama align / panorama cluster) must be run before comparison so that Akin links exist between gene families from different pangenomes. Without them every GFRR score is 0.