Add FOCUS 1.4 schema documentation (upgrade, compatibility, data dictionary)

[flanakin]

🛠️ Description

Per-version schema/column documentation for FOCUS 1.4 (work item W6b) — the docs surface that #2123 deliberately did not cover. All column names and counts were verified against the v1_4 KQL ground truth (IngestionSetup_v1_4.kql, IngestionSetup_v1_4_Datasets.kql, HubSetup_v1_*.kql).

Changes

• docs-mslearn/toolkit/hubs/upgrade.md
  • Added v1_4 to the Data Explorer dashboard compatibility matrix (dashboard 15+ → schema v1_4 → hubs 15+).
  • Added Costs().ProviderName / Costs().PublisherName to the deprecated columns table (retired in 15; replaced by HostProviderName / ServiceProviderName).
  • Extended the per-dataset schema-change table with all 24 Costs v1_4 changes: 22 new columns (7 FOCUS 1.3, 15 FOCUS 1.4 — including the 12 ContractCommitment* per-charge columns with a footnote) and 2 removed columns (ProviderName, PublisherName) with a down-conversion note that _v1_0()/_v1_2() functions still return them.
  • Documented the three new datasets — BillingPeriods (6 FOCUS columns), ContractCommitments (30, using FOCUS 1.4 spec-prefixed column IDs, for example ContractCommitmentBenefitCategory), InvoiceDetails (22) — plus the 5 x_ hub columns each, with the "empty until Cost Management ships FOCUS 1.4 exports" caveat.
  • Noted the v1_2 transforms are deprecated and their update policies disabled as of v15.
  • Fixed two pre-existing "Cost" → "Costs" typos and realigned the table.
• docs-mslearn/toolkit/hubs/compatibility.md
  • Added the FinOps toolkit 15+ row (hubs v1_4, Power BI 15+, KQL reports on the v1_4 schema) and scoped the previous row to 12-14.
  • Kept the export statement accurate: Cost Management exports up to FOCUS 1.2; hubs converts to 1.4 during ingestion.
• docs-mslearn/toolkit/hubs/data-processing.md
  • Updated the transform narrative from v1_0/v1_2 references to v1_4 ({dataset}_transform_v1_4(), {dataset}_final_v1_4, unversioned = v1_4).
  • Added v15 Costs transform notes (provider column mapping incl. fallback chain, FOCUS 1.3 allocation columns, FOCUS 1.4 columns) and new transform sections for the three new datasets.
  • Added the three new raw datasets to the ingestion container dataset list. (The doc doesn't describe KQL script files, so the script split isn't mentioned.)
• docs-mslearn/toolkit/help/data-dictionary.md
  • Added 22 entries for the new Costs columns with concise FOCUS-aligned descriptions and enabled the H quick-link section for HostProviderName.
  • Added a ProviderName entry (removed from Costs in FOCUS 1.4; still in other datasets) and marked PublisherName removed with successor notes.
  • New-dataset (non-Costs) columns were not added — the dictionary doesn't cover non-Costs datasets systematically.
• ms.date set to 07/15/2026 on all four files.

Verification

PowerShell cross-check script (extracts every column name from the doc additions and validates against the v1_4 KQL tables): 111 checks passed, 0 failed —

• upgrade.md Costs v1_4 rows: 24/24 match (22 added exist in Costs_final_v1_4 and not in v1_2; 2 removed exist in v1_2 and not v1_4); doc set == KQL added set exactly.
• New-dataset lists: BillingPeriods 6/6, ContractCommitments 30/30 (spec-prefixed IDs, for example ContractCommitmentBenefitCategory), InvoiceDetails 22/22 — exact match with *_final_v1_4 FOCUS columns.
• All 22 data-dictionary entries exist in Costs_final_v1_4.
• No v1_3 schema references in any modified doc; all markdown tables well-formed (pipe counts validated).

This was an ad hoc cross-check script, not the project's Pester unit suite.

Notes

• Stacked PR: targets flanakin/focus14-phase7-tests-changelog (requires the v1_4 KQL from the stack). Retarget to dev only after the stack merges.
• Does not touch data-model.md (owned by Refresh FinOps hub upgrade procedure for multi-version FOCUS adoption #2123). changelog.md is edited upstream in the stack, not by this PR directly.
• FYI: the 12 ContractCommitment* columns in Costs_final_v1_4 are defined in the FOCUS 1.4 ContractCommitment dataset, not the CostAndUsage dataset (verified against the spec v1.4 tag); docs describe them as per-charge contract commitment context per the shipped schema and changelog.

📋 Checklist

🔬 How did you test this change?

• 🤏 Lint tests
• 🤞 PS -WhatIf / az validate
• 👍 Manually deployed + verified
• 💪 Unit tests
• 🙌 Integration tests

📦 Deploy to test?

• Hubs + ADX (managed)
• Hubs + Fabric (manual) — URI:
• Hubs (manual)
• Hubs (no data)
• Workbooks
• Alerts

🙋‍♀️ Do any of the following that apply?

• 🚨 This is a breaking change.
• 🤏 The change is less than 20 lines of code.

🤖 Generated with Claude Code