Author Name
Admin_niti
Choose Report Type
Publication Date
Report Upload
Download
(2.84 MB)
vertical
Data Management & Analysis
PDF Text
National Data and Analytics Platform: Vision Document 1
NATIONAL DATA AND
ANALYTICS PLATFORM
VISION DOCUMENT National Data and Analytics Platform: Vision Document 3 4 National Data and Analytics Platform: Vision Document 5
• Websites of over 50 Ministries and Departments of the Central Government and data.
gov.in
• Websites of all Departments and Ministries of the State Government, not exceeding 250
(Detailed list of data sources can be found in the RFP titled ‘Selection of Consultant for Development and Operation
of National Data and Analytics Portal’ released on 18.07.2019)
Data access is user-friendly and engaging, tailored to
the needs of policymakers, bureaucrats, researchers,
innovators, data scientists, journalists and citizens alike
Multiple data sets are presented using a standardized
schema, by using common geographical and
temporal identifiers
Standard Operating Procedures are set to ensure
data is updated regularly
User-centric
Coherent
Up-to-date
National Data and Analytics Platform
Vision: Democratize access to public Government data through a
world-class user experience.
Mission: Standardize data across multiple Government sources,
provide flexible analytics and make it easily accessible in formats
conducive for research, innovation, policy making and public
consumption.
Objectives
Data Sources National Data and Analytics Platform: Vision Document 6
India has rich publicly available data. Government departments have digitised various
processes and reporting requirements, which in turn are feeding into management
information systems and interactive dashboards. This data is being used to track and
visualise progress and make iterative improvements on the ground.
However, the data landscape can improve. All public government data should be easily
accessible to all stakeholders in a user-friendly manner. Further, data across sectors should
be coherent to support analysis and dissemination. In addition, there should be easy access
to the most recent data, published reliably.
Overview
Data is not published in a user-centric manner. The current data formats are often not
conducive for research and innovation. Many departments maintain public dashboards with
visualisations, and options to download data in analysable formats. However, some datasets
are only available in PDF, webpage or as an image, making it difficult for further analysis.
If this issue is resolved, researchers and data scientists will significantly save time and
resources in cleaning and preparing government datasets for analysis
Data ecosystem is incoherent due to different standards. Ministries and Departments
do not use a shared standard for common indicators. Attributes like region and time period
defined differently. This makes it difficult for datasets to speak to each other and present a
coherent picture. If we can solve this, many use-cases can emerge. For example, a District
Magistrate can access data across all departments for their district, easily, in one platform.
Challenges
We intend to draw inspiration from the best platforms around the world. Significant
progress in improving user experience while accessing data generated by the public, as
can be seen in platforms such as ‘datausa.io’ and ‘data.gov.sg’. The main utility of such
platforms is to provide a seamless, interactive experience in accessing data across multiple
socio-economic indicators in one place. Also, they provide datasets in a machine-readable
format.
Approach National Data and Analytics Platform: Vision Document 7
We also seek to build on the success of existing Indian data platforms. Over the years,
notable improvements have taken place in the data publishing ecosystem. For example,
data.gov.in provides ready access to data from 165 departments. DISHA, by the Ministry of
Rural Development, provides access to data from 42 schemes across 20 ministries. Various
states also have a ‘Chief Minister’s Dashboard’. Initiatives like these provide a rich source of
data for NDAP.
NDAP will take a user-centric approach to providing access to data. The platform will
be powered by a user friendly search engine, backed by seamless navigation, with a world-
class user interface. Data will be provided in a machine-readable format with customisable
analytics. Dynamic visualisations will be generated based on user search terms. The
platform will be designed and optimised to the needs of policymakers, bureaucrats,
researchers, innovators, data scientists, journalists, and citizens.
NDAP will provide access to data from multiple sectors in one place. Data will be
sourced from different Central and State Ministries and Departments. This will be presented
using a common schema, with clear definitions and narratives to help the user navigate the
differences between similar data from different sources.
Finance
NDAP
DataVisualizationAnalytics
Health
Education
Agriculture National Data and Analytics Platform: Vision Document 8
NDAP will be a reliable platform for up-to-date data. Standard Operating Procedures
(SOPs) will be developed to keep data updated. There will be regular tracking of compliance
to these SOPs. Further, APIs, web-crawling and other methods will be used to keep the
data fully up-to-date.
Success of NDAP relies on effective collaboration. NDAP goes beyond just developing a
platform and will require extensive consultation. Inputs will be needed from data producers
in the government to identify data sources and develop data processing techniques for
the datasets to be coherent. Regular feedback from users -- policymakers, bureaucrats,
researchers, innovators, data scientists, journalists, and citizens -- will also assist in ensuring
the platform stays relevant.
NDAP has the potential to transform data-driven discourse, research, innovation, and
decision-making in India. There are a large number of benefits for up-to-date and reliable
data. NDAP’s vision is to democratise access to public government data and provide a
worldclass user experience. We strongly believe it will be an important tool for the nation’s
progress
Features
NDAP
Facilitate analysis in R,
Tableau, SAS
Data export in
machine readable
format
Common
schema across
sectors
Database of
innovation with
public data
User
friendly search
engine
Social Media
share
Enable cross
sectoral Analysis
Enable easy access to
data from
different sectors
Encourage
discussion on
social media
Showcase innovative
uses and top research
articles for every search
result
FeaturesBenefits Users
Policy Makers
Journalists
Citizens
General Public
Researchers
Academics
Data Scientists
Innovators National Data and Analytics Platform: Vision Document 9
Governance Structure
Technology Vendor
A technology vendor will be engaged for development and operation of NDAP
High Powered Steering Committee
A high powered inter-ministerial steering committee under the chairmanship
of Vice Chairman, NITI Aayog will be set up to provide direction, oversee
progress, guide on data sources, and address various inter-ministerial issues
on collating data.
Technical Advisory Group
A Technical Advisory Group consisting of sector and technology experts will be
established to provide expert consultation. TAG will provide guidance on the
development of the platform, management of data, and aligning the platform for
user-needs.
Project Management Unit
Established at NITI Aayog to coordinate with different stakeholders and
manage various facets of NDAP National Data and Analytics Platform: Vision Document 10
Current members of the Technical Advisory Group include:
• Dr. Mudit Kapoor, Professor, Indian Statistical Institute
• Mr Ajit Pai, OSD, Vice Chairman, NITI Aayog
• Dr Asha Subramanian, Founder & CEO Semantic Web India
• Dr Monojit Choudhury, Principle Researcher, Microsoft
• Dr Lalitesh Katragadda, Founder, Indihood
• Dr Yamini Aiyer, President and Chief Executive of CPR
• Dr Sam Asher, Co-Founder, Development Data Lab
Additional members would be added as the platform matures. Omidyar Network India has
selected IDInsight to provide support to NITI Aayog for this initiative.
We invite you to join us on this journey!
Technical Advisory Group
Way Forward
The first version of National Data and Analytics Platform is proposed to be
released in 2021
NDAP is a pan-India initiative by NITI Aayog. Its success requires extensive
support and cooperation of various stakeholders such as Central Ministries,
State Governments and guidance from experts to realise the vision and
meet the timelines. National Data and Analytics Platform: Vision Document National Data and Analytics Platform: Vision Document 12
NATIONAL DATA AND
ANALYTICS PLATFORM
VISION DOCUMENT National Data and Analytics Platform: Vision Document 3 4 National Data and Analytics Platform: Vision Document 5
• Websites of over 50 Ministries and Departments of the Central Government and data.
gov.in
• Websites of all Departments and Ministries of the State Government, not exceeding 250
(Detailed list of data sources can be found in the RFP titled ‘Selection of Consultant for Development and Operation
of National Data and Analytics Portal’ released on 18.07.2019)
Data access is user-friendly and engaging, tailored to
the needs of policymakers, bureaucrats, researchers,
innovators, data scientists, journalists and citizens alike
Multiple data sets are presented using a standardized
schema, by using common geographical and
temporal identifiers
Standard Operating Procedures are set to ensure
data is updated regularly
User-centric
Coherent
Up-to-date
National Data and Analytics Platform
Vision: Democratize access to public Government data through a
world-class user experience.
Mission: Standardize data across multiple Government sources,
provide flexible analytics and make it easily accessible in formats
conducive for research, innovation, policy making and public
consumption.
Objectives
Data Sources National Data and Analytics Platform: Vision Document 6
India has rich publicly available data. Government departments have digitised various
processes and reporting requirements, which in turn are feeding into management
information systems and interactive dashboards. This data is being used to track and
visualise progress and make iterative improvements on the ground.
However, the data landscape can improve. All public government data should be easily
accessible to all stakeholders in a user-friendly manner. Further, data across sectors should
be coherent to support analysis and dissemination. In addition, there should be easy access
to the most recent data, published reliably.
Overview
Data is not published in a user-centric manner. The current data formats are often not
conducive for research and innovation. Many departments maintain public dashboards with
visualisations, and options to download data in analysable formats. However, some datasets
are only available in PDF, webpage or as an image, making it difficult for further analysis.
If this issue is resolved, researchers and data scientists will significantly save time and
resources in cleaning and preparing government datasets for analysis
Data ecosystem is incoherent due to different standards. Ministries and Departments
do not use a shared standard for common indicators. Attributes like region and time period
defined differently. This makes it difficult for datasets to speak to each other and present a
coherent picture. If we can solve this, many use-cases can emerge. For example, a District
Magistrate can access data across all departments for their district, easily, in one platform.
Challenges
We intend to draw inspiration from the best platforms around the world. Significant
progress in improving user experience while accessing data generated by the public, as
can be seen in platforms such as ‘datausa.io’ and ‘data.gov.sg’. The main utility of such
platforms is to provide a seamless, interactive experience in accessing data across multiple
socio-economic indicators in one place. Also, they provide datasets in a machine-readable
format.
Approach National Data and Analytics Platform: Vision Document 7
We also seek to build on the success of existing Indian data platforms. Over the years,
notable improvements have taken place in the data publishing ecosystem. For example,
data.gov.in provides ready access to data from 165 departments. DISHA, by the Ministry of
Rural Development, provides access to data from 42 schemes across 20 ministries. Various
states also have a ‘Chief Minister’s Dashboard’. Initiatives like these provide a rich source of
data for NDAP.
NDAP will take a user-centric approach to providing access to data. The platform will
be powered by a user friendly search engine, backed by seamless navigation, with a world-
class user interface. Data will be provided in a machine-readable format with customisable
analytics. Dynamic visualisations will be generated based on user search terms. The
platform will be designed and optimised to the needs of policymakers, bureaucrats,
researchers, innovators, data scientists, journalists, and citizens.
NDAP will provide access to data from multiple sectors in one place. Data will be
sourced from different Central and State Ministries and Departments. This will be presented
using a common schema, with clear definitions and narratives to help the user navigate the
differences between similar data from different sources.
Finance
NDAP
DataVisualizationAnalytics
Health
Education
Agriculture National Data and Analytics Platform: Vision Document 8
NDAP will be a reliable platform for up-to-date data. Standard Operating Procedures
(SOPs) will be developed to keep data updated. There will be regular tracking of compliance
to these SOPs. Further, APIs, web-crawling and other methods will be used to keep the
data fully up-to-date.
Success of NDAP relies on effective collaboration. NDAP goes beyond just developing a
platform and will require extensive consultation. Inputs will be needed from data producers
in the government to identify data sources and develop data processing techniques for
the datasets to be coherent. Regular feedback from users -- policymakers, bureaucrats,
researchers, innovators, data scientists, journalists, and citizens -- will also assist in ensuring
the platform stays relevant.
NDAP has the potential to transform data-driven discourse, research, innovation, and
decision-making in India. There are a large number of benefits for up-to-date and reliable
data. NDAP’s vision is to democratise access to public government data and provide a
worldclass user experience. We strongly believe it will be an important tool for the nation’s
progress
Features
NDAP
Facilitate analysis in R,
Tableau, SAS
Data export in
machine readable
format
Common
schema across
sectors
Database of
innovation with
public data
User
friendly search
engine
Social Media
share
Enable cross
sectoral Analysis
Enable easy access to
data from
different sectors
Encourage
discussion on
social media
Showcase innovative
uses and top research
articles for every search
result
FeaturesBenefits Users
Policy Makers
Journalists
Citizens
General Public
Researchers
Academics
Data Scientists
Innovators National Data and Analytics Platform: Vision Document 9
Governance Structure
Technology Vendor
A technology vendor will be engaged for development and operation of NDAP
High Powered Steering Committee
A high powered inter-ministerial steering committee under the chairmanship
of Vice Chairman, NITI Aayog will be set up to provide direction, oversee
progress, guide on data sources, and address various inter-ministerial issues
on collating data.
Technical Advisory Group
A Technical Advisory Group consisting of sector and technology experts will be
established to provide expert consultation. TAG will provide guidance on the
development of the platform, management of data, and aligning the platform for
user-needs.
Project Management Unit
Established at NITI Aayog to coordinate with different stakeholders and
manage various facets of NDAP National Data and Analytics Platform: Vision Document 10
Current members of the Technical Advisory Group include:
• Dr. Mudit Kapoor, Professor, Indian Statistical Institute
• Mr Ajit Pai, OSD, Vice Chairman, NITI Aayog
• Dr Asha Subramanian, Founder & CEO Semantic Web India
• Dr Monojit Choudhury, Principle Researcher, Microsoft
• Dr Lalitesh Katragadda, Founder, Indihood
• Dr Yamini Aiyer, President and Chief Executive of CPR
• Dr Sam Asher, Co-Founder, Development Data Lab
Additional members would be added as the platform matures. Omidyar Network India has
selected IDInsight to provide support to NITI Aayog for this initiative.
We invite you to join us on this journey!
Technical Advisory Group
Way Forward
The first version of National Data and Analytics Platform is proposed to be
released in 2021
NDAP is a pan-India initiative by NITI Aayog. Its success requires extensive
support and cooperation of various stakeholders such as Central Ministries,
State Governments and guidance from experts to realise the vision and
meet the timelines. National Data and Analytics Platform: Vision Document National Data and Analytics Platform: Vision Document 12