Hop atlas home · Overview · Changelog · Architecture · Adding_Data · Introgression · Ribbon · Dosage · Phylogeny · Bundle · Ingest · Ingest_Gbs · Ingest_Wgs · Ingest_Grasdi · Ingest_Rnaseq · Ingest_Assembly

Rendered from hop_atlas/components/DOSAGE.md (2026-10-13)

Dosage atlas (chromosome dosage explorer): how it is built and how to update it

Status: living document, started 2026-10-08. All dosage products are tentative. Part of: the hop atlas web app (hop_atlas/README.md). Served: port 8900, together with the introgression and ribbon atlases. :8906 was retired on 2026-10-08. - Main page: http://10.112.218.205:8900/dosage_atlas/ (redirects to START_HERE.html; cohort views at #ratio). - OSU sex-chromosome panel: …/dosage_atlas/sexpanel/START_HERE.html#sex - Pilgrim genotype-only companion: …/dosage_atlas/pilgrim/index.html

Home: scratch_plots/dosage_atlas/ is self-contained (292 MB). - It was moved out of sync/ on 2026-10-08, at the PI's request. - The former sync/ folders are archived byte-for-byte (with checksum manifests) in /data/hop_data/hop_atlas/dosage_sources/. .MOVED.txt pointers mark their old places.

Tools (hop_atlas/tools/): - atlas_deploy.py (profile dosage) - relocate_dosage.py - check_dosage_http.py

Short answer: can a new cohort or reference go from SRA/BAM to the dosage atlas by following written instructions?

No. - Rebuilding, relocating, testing and deploying the current atlas is scripted (§5). - Everything upstream (manifests, alignment per reference, read counting, allele balance, clade calls) exists only as dated "documentation of what was run", with cohort and reference lists hard-coded in about five places (§2). - Adding a cohort has no documented procedure. - Generic code: the reusable packages annevo_eval/methods/autosomal_dosage (hop_adosage) and annevo_eval/methods/sexchrom_dosage were merged into main in 029bf81 (2026-10-08); the served atlas is not yet built from them.

1. What is shown, and the home layout

• v2 tabs:
• ancestry-aware copy number: per-chromosome CN with intervals; 1 / 2 / 5 / 10 Mb tracks; clade-B read fraction; BAF histograms by class OH / MT / PT / ALL;
• the v1 cohort views: own family, RoR, read share;
• metric definitions.
• Coverage: 5 cohorts (havill, apollo, cascade, saaz, wye) × 6 references (m21 = 21110M, apollo_h1, apollo_h2, saaz_h1, hl1012_h1, cascade).
• X and Y: shown only as depth CN and in the sex panel. The pan-X / European-X reference fix is not in this atlas (sync/2026-10-06/european_x_reference_fix/, annevo_eval/tools/hop_xdosage/).

2. Build chain (each stage is a dated record with hard-coded paths)

Inputs to stage 4: - clade calls: /data/hop_data/multires_clade/{calls,viewer,stage1}; - P2_states.npz; - verified sex: sync/2026-10-04/osu_sex_verification/; - the v1 assets and scripts. Since 2026-10-08 the builder reads these from the archive; build_cn.py, make_page.py, worked_example.py and check_cohort_browser_v2.py were repointed, with backups *.bak_2026-10-08_prehome.

Hard-coded lists to edit for any cohort or reference change: - build_all_cn.sh; - validate_cn.py; - check_cn_browser.py (REFS, POPS); - v1 build_all.py; - mr_config.py (REFS); - build_cn.py (CALLSRC, FRAME, REFHAP, EXCLUDE_BASE={'ERR11608432'}).

What relocate_dosage.py does: - It rewrites only the relative paths that load or link files, and verifies that none pointing outside the home remain. - It re-inserts the Pilgrim companion box if a rebuilt START_HERE.html lacks it. make_page.py does not know that box, which was added by hand on 2026-10-07; before the move, a rebuild would have dropped it.

3. Add a cohort or a reference

There is no tested procedure. From the records it requires all of the following: 1. Add the samples to the 09-25 manifest and run alignment plus counting per reference (stage 2), with GBS rules from annevo_eval/docs/ASSAYS.md. 2. Produce clade calls in multires_clade (annevo_eval/docs/INTROGRESSION_ATLAS_TECHNICAL.md §3–4). 3. Build v1 assets (stage 3). 4. Edit every list in §2. 5. Rebuild, test and deploy (§5).

Cohorts without reads (DArT, e.g. Pilgrim) cannot enter this atlas; link them from the page, as the Pilgrim companion does.

4. Tests

Tests run only when the PI asks (PI, 2026-10-08). They are kept and documented, but not run after every change.

• How the bundled tests run: they use file://, relative to their own folder, so atlas_deploy.py test dosage runs the stage's copies against the stage page, from the stage root. A Chromium crash leaves a multi-GB core file in the working directory, so this keeps it out of the page folder; core files are never deployed.
• Results 2026-10-08: the new home passed all three bundled tests (30 / 30 datasets, 9 presets, sex panel 20 / 20) and the HTTP smoke test on :8900, also after the sync/ originals were removed. The HTTP test fails all four pages on a wrong base URL (must-fail).

5. Stage → build → test → deploy

T=hop_atlas/tools/atlas_deploy.py
python3 $T stage dosage --name my_change      # /data/hop_data/atlas_stages/dosage/my_change/dosage_atlas (copy of the served home)
STG=/data/hop_data/atlas_stages/dosage/my_change
python3 $T build dosage $STG                  # rebuild_and_validate.sh, then relocate_dosage.py into a FRESH home folder in the stage
python3 $T test  dosage $STG                  # the three bundled browser checks -> $STG/checks.txt
python3 $T deploy dosage $STG --dry-run       # files that would change; for text files, lines the served copy would LOSE
python3 $T deploy dosage $STG                 # refuses if the served home changed since `stage`; backs up; deletes nothing
python3 hop_atlas/tools/check_dosage_http.py  # through :8900 after the deploy

There is no separate server any more; :8900 serves scratch_plots/ (see hop_atlas/README.md → Server). Changes to what is served need PI approval.

6. Known gaps

• No runnable upstream path: BAM/SRA → counts → CN exist only as dated records (§2).
• Duplicated lists: cohort and reference lists live in about five files.
• Generic package not used by the served build: hop_adosage is merged (029bf81) but the served atlas is still built by the 09-25 scripts.
• Out-of-scope methods: pan-X / European-X, drHumLupu and the all-13 multi-reference frame are not in the atlas.
• Closed by the 2026-10-08 move:
• the dependence on dated sync/ folders;
• the :8906 server exposing all of sync/;
• the Pilgrim box being lost on rebuild;
• the broken family_shared_site_dosage link;
• the sync/ copy versus /data stage drift (the stage is now only a build input).