Scale your Pandas workflows by changing a single line of code
Modin的外部链接
US，Remote
Modin转发了
Modin 0.32.0 is released - https://lnkd.in/ekxuvVCR. Besides introducing the native query compiler, which is used for more efficient processing of "small" dataframes, there are several performance improvements for running Modin on Ray in a cluster. Check out a recent post on how to scale-out pandas beyond a single node with Modin - https://lnkd.in/eaFuqqZ2. #modin, #pandas, #ray, #dataprocessing, #distributedcomputing
Modin转发了
Excited to announce the general availability of pandas on Snowflake! ?? This is a major milestone since joining Snowflake 10 months ago and I’m incredibly proud of what our team has accomplished. ?? Powered by the Modin open-source project, pandas on Snowflake reflects Snowflake’s commitment to integrating open-source tools and standards with the powerful capabilities of the Snowflake #AI #DataCloud. ?? Since Modin’s inception, our philosophy has always been to meet developers where they are and empower them with the tools they know and love. ?? pandas on Snowflake takes this to the next level by supporting developers to harness the power of Snowflake's AI Data Cloud while keeping the familiar pandas experience. Learn more: https://lnkd.in/gB9mdYWB #Snowflake #DataEngineering #DataScience #Python #OpenSource
Modin转发了
10 months ago, Snowflake acquired #Ponder. Today, we announce General availability for pandas on Snowflake, powered by open source Modin! I'm super proud of what our team has been able to accomplish in such a short period of time. Not only is this an incredible technical achievement, but it also shows a serious commitment from Snowflake to integrate with the open source rather than trying to build a competing ecosystem. I created Modin in 2018. At the time, it was believed that pandas was fundamentally incompatible with databases. Even systems that came after (e.g. Koalas) don't generate SQL, but go through a different dataframe API. Ponder was the first to make this possible, primarily because of our deep understanding of dataframes–we defined the core theoretical dataframe data model and algebra. With this announcement, Snowflake speaks Pandas, and can support workloads at a massive scale. We're not done yet. Stay tuned for more! #python #datascience #dataengineering #data #pandas #dataframe #snowflake #modin
Modin转发了
Excited to announce the general availability of pandas on Snowflake! ?? This is a major milestone since joining Snowflake 10 months ago and I’m incredibly proud of what our team has accomplished. ?? Powered by the Modin open-source project, pandas on Snowflake reflects Snowflake’s commitment to integrating open-source tools and standards with the powerful capabilities of the Snowflake #AI #DataCloud. ?? Since Modin’s inception, our philosophy has always been to meet developers where they are and empower them with the tools they know and love. ?? pandas on Snowflake takes this to the next level by supporting developers to harness the power of Snowflake's AI Data Cloud while keeping the familiar pandas experience. Learn more: https://lnkd.in/gB9mdYWB #Snowflake #DataEngineering #DataScience #Python #OpenSource
Powered by open source Modin!
?? Happy Monday! ?? Excited to share this new video on how you can run distributed pandas at scale directly in Snowflake using our new Snowpark pandas API. ???? https://lnkd.in/g3HFPQ6K #DataScience #DataEngineering #Python
#Polars API for Modin was merged recently. It's still early (and not yet released) but I'm excited about what this will unlock. I think the speed at which we've been able to do this shows the strength of Modin's architecture. This initial implementation supports DataFrame, Series, and GroupBy APIs, ~215 out of ~320 are already implemented and supported. The commit message outlines some of what's left to do: https://lnkd.in/gMBiBE7b There's a lot more to do (and test ??), but it's a pretty exciting time to be in the #dataframe space. Special thanks to Ritchie Vink for early chats about how this might work and Marco Gorelli for early feedback on the prototype. More exciting things are to come, so stay tuned!
?? Heading to #SciPy this week? Come by and say hi ?? the Streamlit booth and grab some exclusive Modin stickers! Plus, don't miss Doris Lee's talk on how you can scale your data science workflows with Modin! ??? Date: July 10 ?? Time: 10:45–11:15 AM ?? Location: Ballroom Details here: https://lnkd.in/gs_3_V-v #SciPy2024 #Python #DataScience #AI #MachineLearning
Modin转发了
I often wonder how much time could be saved if data scientists and data engineers used the same language. That's one of the biggest motivations we had in building Modin. In #Modin, everything is translated into #dataframe #algebra, which means that the execution (e.g. Snowflake) can stay the same, while the #API (e.g. #pandas, #polars) can change.
Modin转发了
RunLLM is now on the Modin_project documentation site — thanks to Devin Petersohn & the Modin team for their help! Let us know what you think: https://lnkd.in/eZw23raW