In astrophysics, dark matter refers to the invisible substance that determines the organization of galaxies on grand scales. We can't observe it directly, but know it is essential for our calculations to make sense. At Ensemble, we borrow this idea to describe what our product does for machine learning. When building ML models, we work with data that tends to be sparse, statistically complex, or low sample size (or all three!). Traditional feature engineering captures the obvious signals, but there’s always hidden, unobserved information — like "dark matter"— that your model overlooks. This is where Ensemble’s breakthrough comes in. Using novel statistical theory, we identify and extract new, unlabeled variables that are custom-derived to enhance the performance of each of your specific models. What makes these features special is that they are ???????????? ???????????? ????????????????????, meaning they add unique and meaningful signals without redundancy — all derived from the data you already have. The result? Better predictions from the same models and datasets, whether you’re using deep learning or a simple linear regression. With just three lines of code, you unlock a new level of data richness and predictive power. Let’s start thinking beyond what’s visible in our datasets and start leveraging the "dark matter" of machine learning. ??Could your models benefit from discovering their hidden dimensions? #MachineLearning #DataScience #FeatureEngineering #PredictiveAnalytics
Ensemble的动态
最相关的动态
-
For more than 20 years in experimental particle physics and astrophysics, machine learning has been accelerating the pace of science, helping scientists tackle problems of greater and greater complexity. https://lnkd.in/e8BGNwDV #AI
-
?? For more than 20 years in experimental particle physics and astrophysics, machine learning has been accelerating the pace of science, helping scientists tackle problems of greater and greater complexity. More: https://bit.ly/4aTxsAG #machinelearning #AI
-
In the past week, I revisited one of my past projects where I used pretrained CNNs for Galaxy Morphology Classification. This led to an upgraded project focused on anomaly detection to identify unusual galaxies, which successfully flagged interacting systems and samples with imaging artifacts in the testing set! I’ve just finished writing a blog where I share my astronomical perspective (to the best of my knowledge) on the results and interpretations. Check it out here if you're interested ??: https://lnkd.in/geavZtkd
-
As a passionate space lover and data science enthusiast, I recently came across an insightful article discussing the powerful blend of AI, machine learning, and astronomy. It explores how these technologies are transforming the analysis of astronomical data and improving predictions. A must-read for anyone interested in the future of space exploration and data science!" You can end with the link to the article: https://lnkd.in/gNeApaxu #DataScience #MachineLearning #ArtificialIntelligence #DataScienceInSpace #MLinAstronomy #SpaceTech
-
My SEFRI funded ERC took a big step today. We got our AI model that can distinguish dark matter from cosmic noise published in Nature Magazine ( astronomy). This is a big step to finally understanding what dart matter is. Next step applying to data! https://lnkd.in/ejg3jrr5 ( full article here : https://rdcu.be/dTa4m )
-
The National Science Foundation (NSF) is launching two new AI programs aimed at advancing astronomical sciences by developing novel algorithmic capabilities. In collaboration with the Simons Foundation, these AI institutes will partner with academic researchers to create AI software capable of processing complex astronomical data, including images, text, code, and spectra, which are currently beyond the scope of standard AI tools. These initiatives represent the first time NSF’s AI Research Institutes will be focused on astronomy, addressing key research areas like dark matter and the origins of life in our solar system. The NSF-Simons AI Institute for Cosmic Origins, led by the University of Texas at Austin, will simulate stellar chemical processes, while the NSF-Simons AI Institute for the Sky will tackle astrophysics challenges like neutron star and black hole physics. Researchers aim to build cutting-edge AI algorithms, including large language models trained on existing astronomical research, to bridge the gap in current AI capabilities. These efforts are set to revolutionize how astronomical data is processed and analyzed, driving advancements in the field. #ai #astronomy #nsf #simonsfoundation #astronomicaldata #research #cosmicorigins #astrophysics #blackholes #darkmatter Source- https://lnkd.in/gEms9ZVb
-
-
I recently had the opportunity to attend the workshop organized by the Inter-University Centre for Astronomy and Astrophysics (IUCAA) on AI/ML Applications in Astronomy and Astrophysics. The workshop started with a talk on introduction to Generative AI and Large Language Models (LLMs), covering foundational concepts such as RNNs, LSTMs, and Transformers. It included multiple talks from AI experts and PhD scholars discussing applications of AI/ML techniques to various domains in astronomy and astrophysics including morphological classification of galaxies, accelerating cosmological simulations, gravitational waves, spectral classification, Solar astronomy and all aspects of time domain astronomy. It also included engaging hands-on sessions, and it was concluded with thought-provoking panel discussion on Future and Sustainability of AI. As someone working at the intersection of astronomy and data science, I found all the sessions extremely useful. I gained several insights from this workshop that can be applied to my research project. That includes: 1. Using Monte Carlo dropout methods to quantify uncertainty in the model's predictions. 2. Leveraging GANs and diffusion modeling to achieve higher resolution, with reionization parameters as conditional modifiers. 3. Applying GradCAM analysis to improve model explainability. A huge thanks to IUCAA, Pune, India for organizing this workshop and to all the speakers who shared their expertise. #IUCAA #Astronomy #Astrophysics #Cosmology #ArtificialIntelligence #DataScience #Research #AstroML #DeepLearning #MachineLearning #GANs #GradCAM #Transformers #LLMs #NLP #AISustainability
-
Symmetry magazine covered the story of my experience creating the largest catalog of Low Surface Brightness Galaxies (to date) during my PhD. This catalog is currently used (among other things) to train Machine Learning models that automate the process of discovering galaxies (and thus eliminating the need to grab a Cappuccino and inspect astronomical images for hours - for better or worse!) !! Thanks to Aleksandra ?iprijanovi? , one of my mentors at the time! #MachineLearning #Astrophysics #AI
-
Counting down to the first day of IndabaX SA ???? 2024! ?? We're thrilled to announce our inclusive Research Hackathon at Deep Learning IndabaX SA ???? 2024 in?partnership with BRICS Astronomy Magazine, ilifu and Hack4dev. Whether you're new to AI or a seasoned student researcher, this is an exciting event. ?? Challenge: Galaxy Classification ??? Problem Release: 5 July 2024 ?? Main Hackathon: 11 July 2024 Team up with brilliant researchers, scientists, and engineers to unravel the mysteries of the cosmos. Get a head start this week for your chance to be crowned the Guardian of the Galaxy and claim the grand prize of R5000! Don't miss this opportunity to push the boundaries of AI and astrophysics. Join us and make your mark in the universe of machine learning! #IndabaXSA #IndabaXSA2024 #AIResearch #GalaxyClassification #MachineLearning
-
Turing AI Fellow / Professor of Radio Astronomy / Co-Director of Policy@Manchester, University of Manchester
It was a pleasure to speak last week at the European Commission Mutual Learning Exercise on advancing national AI policies in science. Many thanks to Jessica Montgomery and the ai@cam team for leading this event. In particular, I spoke about the development of foundation models for science, with a focus on astrophysics, and the different implications that has for infrastructure policies - both in terms of model development and model exploitation. My key reflections on this topic are: 1. foundation models are probably best "owned" by the same collaborations that own the data*: this can enhance the usability of the data, potentially democratising the use of otherwise unwieldily data volumes (astronomy and particle physics in particular have a big big data problem...), it also enforces traceability, supports reproducibility of downstream research, and ensures that domain expertise is encoded into the models; 2. the same access to specialised (e-)infrastructure that is required for development of foundation models is not the same as is required for their exploitation: i.e. training foundation models requires very different compute from the services built around them for users, and if even fine-tuning can be done remotely / via API services then the compute models will look different; 3. the integration of foundation model based services for science with other agentic AI is likely to require federated services, in particular around AAAI and data management, to enable their use with proprietary scientific data. There is also a growing interest in multi-modal foundation models [e.g. AstroCLIP**, AstroM3***] in astronomy - unsurprisingly since we have so many modalities/sources of data that contribute to different analyses. Fortunately, there are many astronomical archive providers that support diverse modalities and could be used as test cases for general purpose multi-modal foundations. [*] I'm using "owned" here to imply "take responsibility for" rather than "control access to" - although at some level these are synonymous. Note that this does not preclude any collaboration from developing methodologies or techniques that could be adopted by / incorporated into an archival foundation model. [**] AstroCLIP: https://lnkd.in/e-syMQEG [***] AstroM3: https://lnkd.in/eafJAFMR
Créer des marques qui séduisent les utilisateurs | Entrepreneur x Designer x Créateur de contenus positifs | Co-Fondateur Mapossa x Co-Fondateur @LePerfectionniste x Fondateur ThinkGood. | Découvrez le Brand Canvas
2 个月Steve Jobs said creativity is the ability to connect concepts from different fields to create something new, and I can clearly see it here. Even for a non-tech savvy it's clear how deep you guys went into your product and study. My sincere appreciations, keep up ??