Document Procesing Tracker

Orgpedia Document Processing Tracker

Follow how government documents are transformed into accessible, usable data — from document ingestion and machine-powered extraction to the creation of data products and easy-to-use websites..

Funnel Ingestion

185,248

Documents Processed

Total volume of government documents downloaded, organized and made available on Github, Huggingface and Archive.org.

AI Extraction

104,509

Information Extracted

Total volume of documents from which structured records and relevant entities are extracted and translated.

Public Presentation

102,704

Information Presented

Volume of documents whose entities are presented on websites.

Documents Processed: Detailed Breakdown (185,248)

Source collections, formats, and OCR pipeline throughput.

Collection / Source	Issuing Authority	Time Range	Estimated Docs	Status
Cabinet Secretariat (cabsec)	Cabinet Secretariat, Government of India	1947 – Present	904 docs	Upto date, Manual upload
Maharashtra Government Resolutions (mahGRs)	Mantralaya, Govt of Maharashtra	1961 – Present	173,098 GRs	Upto date, Daily upload
IAS Civil List(iasCivilList)	Department of Personnel and Training, Government of India	1955 – Present	69 docs	Upto date, Manual upload
Municipal Corporation Circulars (punePMC)	Pune Municipal Corporation	2018 – Present	9332 circulars	Upto date, Daily upload
Maharashtra Legislature Starred, and Unstarred Questions(mahmls)	Mahrashtra Legislature	2012 - Present	1,845 docs	Upto date, uploaded after session

Ingestion Pipeline Details:Each document is downoaded, it is attached the metadata from the site and then uploaded on wayback machines, archive.org and huggingface as dataset. The code that does this is available on Github.

Information Extracted: Detailed Breakdown (104,509)

Relevant entities are extracted from documents and then translated

Collection / Source	Documents	Entities	Translations	Status
Cabinet Secretariat (cabsec)	904 docs	4,282 tenures, 927 ministers	22 languages	Upto date (Oct 2026)
Maharashtra Government Resolutions (mahGRs)	101, 800 GRs	order number, locations and INR amounts on all	translated to English	Upto date
Maharashtra Legislature Starred, and Unstarred Questions (mahmls)	1,845 docs	745,916 questions	translated to English	Dec 2025

GovDoc Layer Architecture:In GovDoc 3.1, every entity span is permanently anchored to words on the page. Line builders, block builders, and table finders group words without destroying entity provenance, allowing extraction tools to be upgraded without breaking downstream entity links.

Information Presented: Detailed Breakdown (102,704)

Verified public knowledge, interactive timelines, and multi-lingual access.

Flagship PortalLive

Ministers of India (1947–To Date)

Central Cabinet Secretariat (1947–Present)

Documents904 docs

Entities3,885+

Explore Council of Ministers →22 Languages

State GovernanceWeekly Monitor

Weekly Summary of Maharashtra Government Resolutions (GRs)

Mantralaya, Government of Maharashtra

Documents173,098 GRs

Entities—

Open Maharashtra GR Summary Portal →Weekly Updates

Open Data Deliverables:In addition to web portals, data is packaged for developers via Python package (`pip install orgpedia_cabsec`), CSV summaries, and JSON schema-validated outputs in GitHub repositories.

Processing Milestones & Roadmap

✓

Milestone 1: Central Cabinet History (1947–2026)

Full Council of Ministers from Independence to present day across 22 languages.

Completed

✓

Milestone 2: 250,000 Documents Ingested

Maharashtra GR collection + Cabinet Secretariat orders + indicGovBench release.

Current (Live)

○

Milestone 3: 500,000 Documents & Multi-State Expansion

Ingesting state gazettes for Karnataka, Gujarat, and Tamil Nadu.

Target Q2 2027

○

Milestone 4: 1,000,000 Documents & Sovereign Indic DPI

National-scale gazette indexing and community review studio deployment.

Target Q4 2027