paris_habitat_integration_report.md

Paris Habitat → Stonal Integration Report

Organization: KIIIIIKS (dev) Environment: api.stonal-dev.io Source CSV: paris_habitat_20250425.csv — 210,913 rows, semicolon-delimited, Latin-1 encoding External ID source: PH Properties source: PH Date: 2026-02-14

---

Asset Summary

Asset Type	Count	External ID Pattern	Parent	Properties
BUILDING_GROUP	1,461	{HPCDPAT2} (e.g. 12DO)	—	ADDRESS, POSTAL_CODE, CITY, LAT, LNG
BUILDING	3,844	{HPCDPAT2}-{HPCDPAT3} (e.g. 12DO-G)	BUILDING_GROUP	ADDRESS, BUILDING_DATE, LAT, LNG
BUILDING_SECTION	10,926	{HPCDPAT2}-{HPCDPAT3}-{HPCDPAT4} (e.g. 12DO-G-03)	BUILDING	—
LEVEL_SECTION	49,934	{...}-{J7NIVETAG} (e.g. 12DO-G-03-4)	BUILDING_SECTION	—
RENTED_UNIT	126,495	{GGNOUG} (e.g. 009516)	LEVEL_SECTION	SHA, SURFACE_UTILE, TYPOLOGIE_DE_LOT, USAGE_LOT
PARKING	51,450	{GGNOUG} (e.g. 207650)	LEVEL_SECTION	TYPOLOGIE_DE_LOT
Total	244,110

Hierarchy

BUILDING_GROUP  ←  HPCDPAT2 (site code)
    │               1,461 sites
    │
    BUILDING  ←  HPCDPAT2 + HPCDPAT3 (building within site)
        │           3,844 buildings
        │
        BUILDING_SECTION  ←  HPCDPAT2 + HPCDPAT3 + HPCDPAT4 (staircase/entrance)
            │                    10,926 sections
            │
            LEVEL_SECTION  ←  ... + J7NIVETAG (floor number)
                │                  49,934 floors
                │
                RENTED_UNIT  ←  GGNOUG (where JICDTNATUG = LGT)     126,495
                PARKING      ←  GGNOUG (where JICDTNATUG = PKG)      51,450

RENTED_UNIT and PARKING parent assignment is conditional:

• If floor (J7NIVETAG) is present → parent = LEVEL_SECTION
• If section (HPCDPAT4) is present but no floor → parent = BUILDING_SECTION
• If only building (HPCDPAT3) is present → parent = BUILDING

Property Mapping

BUILDING_GROUP

Stonal Property	CSV Column	Example
ADDRESS	HP2_MPADRPORUE	10 rue Dorian
POSTAL_CODE	RCCDPOSTAL	75012
CITY	RCLIBCOM	PARIS
LAT	Lat_site	48.846741
LNG	lng_site	2.392702

BUILDING

Stonal Property	CSV Column	Example
ADDRESS	MPADRPORUE	10 rue de Picpus
BUILDING_DATE	HPDTECSTR	1950-08-01 (normalized from DD/MM/YYYY)
LAT	Lat_bat	48.847446
LNG	lng_bat	2.393195

RENTED_UNIT

Stonal Property	CSV Column	Example
SHA	GTQTEFACT_SHA	39,29
SURFACE_UTILE	GTQTEFACT_SUT	39,29
TYPOLOGIE_DE_LOT	K3LBLTYPUG	Type 2
USAGE_LOT	JILBLNATUG	LOGEMENT

PARKING

Stonal Property	CSV Column	Example
TYPOLOGIE_DE_LOT	K3LBLTYPUG	Box extérieur

Send Results

Run 1 (2026-02-13, ~2h18m)

• Batches: 12,209 (batch size: 20)
• Success: 11,224 batches
• Failures: 985 assets across all types
• Root cause: 4 buildings with apostrophes in codes (19DJ-S', 19BP-Y', 19BT-Y', 14V8-A') rejected by API identifier validation, cascading to all their children

Run 2 (2026-02-14, ~2h40m) — after fix

• Batches: 12,209
• Success: 12,209 batches (100%)
• Failures: 0

Issues Encountered & Fixed

Issue	Root Cause	Fix
LATITUDE property rejected on BUILDING_GROUP	Property only valid for BUILDING, not BUILDING_GROUP	Changed to LAT (valid for both)
LONGITUDE property rejected on BUILDING_GROUP	Same as above	Changed to LNG
CONSTRUCTION_DATE property rejected	Property code doesn't exist	Changed to BUILDING_DATE
TYPOLOGIE property rejected on RENTED_UNIT	Property code doesn't exist for this type	Changed to TYPOLOGIE_DE_LOT
USAGE property rejected	Property code doesn't exist	Changed to USAGE_LOT
Date format DD/MM/YYYY rejected	API requires YYYY-MM-DD	Added normalize_date() conversion
Apostrophes in building codes (S', Y', A')	API only accepts [a-zA-Z0-9+\-_/.]	sanitize_id() now replaces any special char with -
SUT, STATUS, FINANCING, DOOR_NUMBER rejected	Property codes not configured for this org	Removed (replaced SUT with SURFACE_UTILE)

What's NOT imported

• ~22K technical components (JICDTNATUG = ETG, ASC, CHF, BOU, ENV, EXT...) — these have no standard Stonal asset type and often lack floor information
• Organizational hierarchy (columns ORCDORG1-3) — describes management structure, not physical assets
• Surface chauffée (GTQTEFACT_SCH) and surface réelle (GTQTEFACT_SRE) — no matching property codes identified
• Financing category (J1CDTCATUG/J1LBLCATUG) — property not configured on this org
• Occupancy status (J8CDTETAUG/J8LBLETAUG) — property not configured on this org
• QPV / co-ownership / accessibility flags — property codes not verified

Technical Details

• Code: paris_habitat_sync/ in exp-ideas repo
• Stack: Python 3.10, requests, uv
• CSV encoding: Latin-1 (not UTF-8)
• Column renames: Etage name → Etage_name, Code _bat → Code_bat (spaces in CSV headers)
• Batch size: 20 assets/batch (send), 500 assets/batch (receive)
• API endpoint: POST /datalake/v1/organizations/KIIIIIKS/assets