Big Data in India can help in overcoming the country’s main challenges

Big Data in India can potentially alleviate major challenges faced in the field of education, health, energy, crime, national security, employment, food production and governance.

big data in india

Big Data has already seen success in its initial use to solve various challenges which countries are facing

Big Data refers to collection of huge data sets (in the range of exabyte) which are so complex and unstructured that they cannot be analysed using a traditional data processing software. Traditional data is very structured i.e. it fits into neat columns in MS Excel/ MS Access which are of defined format. Such data can be easily analysed using off the shelf computer software.

Now think of India and all the information which is been generated every second by its billion inhabitants. Daily there are reports getting generated by various government agencies. The reports pertain to crucial information on health, education, social programs, government spends, crime, GDP, industry, agriculture to name a few. These reports are either on paper files or stored on offline department computers in various formats. Then there are private enterprises generating petabytes of data based on consumer online shopping behaviour, credit card usage, online social media updates or simple phone calls. Add NGO and individual initiatives which monitor jungle depredation to potable drinking water in remote villages. Indians are generating unfathomable amounts of data in varied formats using different devices at different speeds of generations. This gargantuan data breeding animal makes for a strong case that  Big Data in India can alleviate its myriad challenges. Let’s for a moment assume that we can make all this data available online and available to Big Data processing softwares. In such a world, what can we do in various sectors using Big Data in India?

Big Data in India
In India there is disparity between what skills education offers and what skills are needed for available jobs

 Education:  In India the unemployment rate has increased. Meanwhile the private education sector is booming. However as students pass out and do not get jobs, this sector too will turn less lucrative in the near future. Now assume we have different sectors, from private construction companies to government PWD department, job data available online with details like how many people are employed, for what positions and the salary bandwidth. Government websites will have data on NREGA and other employment opportunities and job sites will have private sector jobs will corresponding qualifications required. Education department has information on how many students have which qualifications along with the curriculum they have mastered. Big Data applications can then match the job opportunity with what the education sector is teaching. This will lead to more relevant on job training, help students decide better what stream to pursue other than “engineering and commerce”, updating of school curriculums and direct freshers to the real locations where job opportunities are available.

big data in india
Crop production in India out of sync with demand patterns and weather impact forecasts

Food production: Fresh on everyone’s mind is the case of onion prices shooting through the roof and images on prime time TV showing affected home makers. Today, farmers are at the mercy of the weather and have no visibility to future demand predictions across India for their produce. Notwithstanding the fact the sector requires massive government policy changes, Big Data application can make dramatic changes. Weather pattern data from meteorological department can be paired with wholesale market sales data to derive which crops will be in shortage. So if heavy rain is predicted in Punjab, then UP can shift from rice to wheat for a crop cycle as there is a high possibility of Punjab wheat crop getting affected. Punjab farmers can shift to rice or green vegetables which are more resistant to rain. The fallout will be more stable wheat prices and better profits for the farmers of both the states.

big data in India
Today police and security forces react only when the crime has taken place. By then it’s too late for the victim

Crime: The historical background of all criminals from fraudsters to petty thiefs to eve teasers to murderers to terrorists is spread across police stations, military and court archives. The location of known criminals, along with latest crime reports mapped to the socio-economic condition of various localities can help todays police force deploy patrols much more effectively. In fact this has already been successfully applied in Memphis USA to effectively patrol the city and the program is very popular with the citizens.  Another effective use case, would be monitoring real-time sale of liquor in a city’s pubs and directing traffic police to those locations.

big data in india
Energy generation and distribution companies monitor only real-time supply and demand. There is minimal planning in forecasting the situation leading to grid collapse at its worst

Energy: Last year the of Northern, Eastern and North Eastern grids collapsed in July plunging 620mn people into the darkness. This can be avoided if stakeholders can execute proper planning aided by Big Data. Electricity meters from every single household, shop and factory mapped with weather conditions will forecast demand surge. Forex rate, bank lending rates, diesel prices will forecast more demand from the industrial sector. Housing sector data will identify future demand areas. Big Data in this way will forecast short term and long term demand patterns. Similarly rain and snow melt data mapped with developments which impact crude production overlaid with the state of the generation and transmission infrastructure will predict challenges on the supply side of the energy story.

big data in india
Different hospitals and state health care centers do not share information among themselves to predict possible epidemics

Epidemics: India boasts about having the longest list of diseases which can lead to epidemics. Lots of cases of dengue or malaria get treated at private hospitals and the information stays there. If every single case is registered online as patients got admitted then based on historical disease occurrence in severity and time along with weather patterns and drainage situation, Big Data applications can detect zones at high risk of epidemics. Municipalities and rural healthcare departments can send fumigation and health inspection teams to these areas to reduce the probabilities of breakout. They can make sure that all hospitals and rural health care centers have adequate medicine stocks for treating afflicted people. This logic can be applied to veterinary sciences for livestock epidemic monitoring too.

Tax, Inflation, Governance, National Security all can derive similar benefits from Big Data in India. Successful use cases in each of the above mentioned sectors have been demonstrated by governments of many developed countries. Execution of Big Data in India will help its citizens lead a more stable and fulfilling life.

However there are major challenges to adoption of Big Data in India. Transparency will not be welcome by many vested groups whose power and money comes lack of information. India, tied down with bureaucratic inefficiencies, lack of funds for IT infrastructure developement and data sciences staff will take a long time to execute Big Data. The first major successes will come from large private enterprises in banking, telecom, payment, e-commerce and social marketing using the expertise of Big Data Startups in India. Yet there is hope that some visionaries in the government will create pools of excellence of Big Data in India.


2 thoughts on “Big Data in India can help in overcoming the country’s main challenges

  1. kidakaka

    I had written glorious things about Big Data in our college’s magazine (Management Canvas), you can read it here –

    However as time goes by … we also have to be cognizant about the fact that very few providers are around who can do justice to this term and we are a few years behind before Big Data can be taken mainstream.

  2. Pingback: Big Data Companies in India

Leave a Reply to kidakaka Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>