Nasa's massive big data image library available to researchers by metadata and cloud first approach.
Everyone is now able to access NASA's online content featuring more than 140,000 images, audio files, and videos after the migration of the content platform to AWS.This is great example where it becomes quite evident that how massive data sets availability can scale only on a cloud platform.
NASA's AWS Cloud Solution collects data from 60 centres into Amazon cloud and generates a fast searchable user interface for researchers and students worldwide.
NASA took help from contractor InfoZen, which helped the space agency in migrating 200+ websites and applications to the cloud in recent years through its Web Enterprise Service Technology contract. The agency helped NASA to create a common metadata system. In order to perform meaningful search across multiple platforms, metadata centralization was the approach that was taken. The result is a awesome , scalable and fast loading library located here : https://images.nasa.gov
My coaching to the big data architects, engineers and data scientists building massive big data applications will be to understand this use case and implement metadata first approach while building big data ecosystems or enterprise data lakes. The win also lies in building the infrastructure in cloud rather than bare metal data centres. Nasa had tried several approaches traditionally in mid-2000 but the winning came by the cloud route.
Feel free to comment on the thread , if you have driven metadata and cloud first approach in your organization to scale big data workflows.
About the Author :
Vin Patel is a Canadian technology author with years of experience in technologies like Big Data and advanced digital marketing. He is an open source innovator building series of fault tolerant,large-scale, high-availability applications for large enterprises in Canada . He is a mentor to several Canadian technology startups on open source technologies.
Reference : https://www.nextgov.com/cloud-computing/2017/04/secret-ingredients-nasa-used-builds-its-huge-photo-library/137161/?oref=ng-HPriver
Forensic Accountant
7 年I wish NASA had been established by our stupid Mullas who say for every new discover that " everything is in Quaran", but have never discovered a loaf of bread even and growing their shameless tummies on Zakat and Jews sent charity! Good deeds should be admired and appreciated irrespective of the doers.
Helping Engineering Teams Release Frequently & Fearlessly!
7 年also look at youtube video data dump with meta data.