Big Data: Definition, Importance, and the Case for Equity

“Data” is a term that we are all very familiar with. It refers to information, mostly numerical, stored, or processed by your computer or mobile phone. This includes text, emails, images, documents, audio, video, or applications, and much more. Basically, data is stored in the form of 0’s and 1’s and is processed and saved digitally.

Error! Filename not specified.

We are in a world that thrives so much on data and technology. Honestly, it’s safe to say that data is the new gold or oil. We consume so much data regularly, and it’s not about to stop. More than 95 million photos and videos are shared on Instagram daily, and 306.4 billion emails are exchanged. We generate about 2.5 quintillion bytes of data every day, and it is projected to grow to 463 exabytes of data by 2025. Massive, right? Yes! This makes it really difficult for it to be processed by traditional systems. This data is known as Big Data. 

Big data simply refers to a large amount of exponentially growing data that cannot be processed, analyzed, or stored using traditional software. These data can be structured, semi-structured, and unstructured.

Big data can be described using 3Vs. It is enormous (volume), of diverse sources (variety), and is continuously growing at an uncommon speed (velocity). Recently added characteristics include veracity (accuracy and trustworthiness of generated data), variability (unpredictable data flows), and value (benefits that come from analyzing the data). 

So, what good does Big data actually do? Is it just some collection of enormous and complex data, or does it actually hold immense value? Big data is very important to many businesses and companies because of the valuable insights that can be gained from it and the competitive advantage it gives them in the market. It’s not just about the amount of data, what really matters is what you do with it. Big data processing and analysis requires open source big data such as Apache Hadoop, Apache Spark, etc. 

Companies can use big data to improve decision-making, boost efficiency, improve customer experience, develop new products, create personalized marketing campaigns using individual preferences, and increase profits. 

A real-life example of how it applies to the healthcare industry is that hospitals around the world generate a large amount of data in the form of medical records at a very high speed. This data can be in excel format or x-rays. Analyzing this data would be beneficial to the industry and lead to reduced costs, improved patient care, and early disease detection. 

Another example is, streaming services such as Netflix or Hulu collect user behavior data from customers. Using their data, they can gain insights on customers' preferences and make recommendations based on that data, thus improving customer's experience. Big data can also be useful in nearly all industries, from marketing, business, and e-commerce to education, agriculture, finance, and sports.

Thanks to Big data, there are many promising career paths you can get into.  Whether it's being a data scientist, a machine learning scientist, a BI developer, or a data Analyst. These career choices require some level of dedication and commitment to your paths. They are several skills that you need to learn to become a data scientist. They include primary technical skills like math and statistics, programming and domains/business knowledge, and soft skills such as leadership, time management, and communication.  

Despite the growing prominence and benefits of big data, racial and gender inequalities still thrive, especially for black women. At the basic level, there is an underrepresentation of black women in STEM-related courses and low mentorship opportunities, and at advanced levels, recruitment and cultural exclusion in the workplace. According to recent estimates, only 3% of black people are accounted for in the Data and Analytics community with only 27% of women in the US represented in data science roles. 

There are concerns that this may increase the risk of racial discrimination and gender inequalities. Big data may not always be completely accurate due to the large volume and variety of data collected and the velocity at which they are collected. There may be underrepresentation or misrepresentation, which may exacerbate inequalities. One system technology used to assess the future risk of re-offending among defendants was shown to discriminate against black people. 

In spite of this, big data also offers the opportunity and leverage to promote inclusion and equity. Mitigating inequalities would require that companies and businesses ensure they are diverse as much as possible in terms of leadership, project teams, and recruitment teams. Promoting transparency and a work inclusion culture would also create an atmosphere of inclusion and diversity. 

At the basic level, girls need to be exposed early to STEM-related skills. Also, they lack exposure to positive female role models in STEM and as such, they may shy away or be eventually pushed off course by gender stereotypes. Positive awareness and recognition of key black women in data science could inspire and encourage many females to pursue these roles. 

Key black women promoting equity and inclusion in a data science field and beyond are Rediet Abebe, Dominique Harrison, Jordan Harrod, Muthoni Wanyoike, Ruth Agbakoba, Ruha Benjamin, Yeshimabeit Milner, Timnit Gebru, and Joy Buolamwini. These women are smashing gender stereotypes and can positively influence female students to pursue STEM roles. 

If we want to get more girls and women, especially black and minority ethnic (BME) girls, into Big Data roles, we need to be really proactive in ensuring they get the necessary knowledge and empowerment via education, active mentorship opportunities, collaborative efforts, hackathons, and related events. 

The #BeMe project is a major player in promoting gender equality in technology and empowering BME young women to pursue STEM careers. This campaign was launched to give one million young women access to education and mentor them in STEM roles via the Academy Achievers’ program.

There are plenty of opportunities in STEM areas, and more women from diverse backgrounds can benefit from them. They need access to several technical skills and positive role models that can support them and elevate their confidence.