Apache StreamPipes

If you have any good StreamPipes success story or technical extension you want to share, please consider to submit a talk to this year's ApacheCon!

?? Apache StreamPipes 0.97.0 is here! ?? The latest release brings powerful new features to make Industrial IoT data processing more flexible, secure, and user-friendly. Here’s what’s new: ?? New Asset Browser A new Asset Browser makes it easier than ever to structure and discover StreamPipes resources such as pipelines, dashboards and connectors and link them to physical assets. ?? Pipelines & Adapters as Code Define and manage adapters and pipelines programmatically using YAML or JSON. ?? Enhanced Security & Connectivity OPC UA Certificate Support: Secure connections with certificate-based authentication. Kafka SSL/SASL Authentication: Support encrypted communication with Kafka brokers. OAuth 2.0 Integration: Connect StreamPipes with external authentication providers ?? Data Explorer Improvements Added more configuration options to visualizations in the data explorer. New data explorer visualizations such as traffic light and gauge. Harmonization of data explorer and live dashboard. ? Usability & Stability Enhancements Improved workflow to modify running pipelines. Better data preview for connected data sources. More intuitive pipeline details view. In total, this release includes 159 fixes and improvements, making Apache StreamPipes even more powerful for your IIoT and data streaming needs! #ApacheStreamPipes #IIoT #InternetOfThings #OpenSource #Manufacturing #DataAnalytics #AIoT #OSS The Apache Software Foundation

?? Feature Preview: Organizing industrial assets with StreamPipes! ?? While we just released Apache StreamPipes version 0.95.0 in June, we’ve been busy working on the upcoming 0.97.0 release since March. Scheduled for later this year, the new release will introduce a bunch of new features designed to simplify the management of industrial data sources. You'll find it much easier to connect physical objects like production lines and machines with StreamPipes resources, including data connectors, pipelines, and data views. With the upcoming version, assets can be organized hierarchically, tagged with labels, and assigned to specific locations and sites. The enhanced asset browser will also let you conveniently filter resources (e.g., by location, by asset type or by label) ??. Check out some of the screenshots below—more updates to come! ?? #IIoT #DataManagement #OpenSource

?? First Apache StreamPipes Meetup in China ?? A few weeks ago, the first-ever Apache StreamPipes meetup in China took place! Our PMC members 罗振羽 and Zike Yang introduced the open source project, explained technical foundations and presented live demos. A huge thank you to everyone who joined us and contributed to this event - below are some impressions. Great to see how the StreamPipes community gains worldwide traction - looking forward to seeing how Apache StreamPipes continues to transform Industrial IoT data into actionable insights. #ApacheStreamPipes #IIoT #Analytics #Meetup #China #OpenSource

We highly recommend Alessandro Albani's article on a vulnerability we successfully resolved in our latest release, version 0.95.0. While it's never easy to be highlighted as a negative example, we're grateful for the opportunity this has provided to strengthen StreamPipes, making it more robust and secure. In the end, this experience underscores one of the greatest strengths of open source: fostering collaboration among diverse experts, leading to improvements that wouldn't be possible otherwise. Thank you very much for the great collaboration, Alessandro! #OpenSource #CyberSecurity #Collaboration #SoftwareDevelopment #StreamPipes #ASFWay

We are thrilled to announce the immediate availability of Apache StreamPipes 0.95.0! ?? This release addresses over 200 issues, introducing numerous improvements and bug fixes to enhance the user experience significantly. ?? Here are some highlights of the latest release: ?? Enhanced Integration with Apache PLC4X StreamPipes 0.95.0 now features integration with PLC4X using protocol metadata, expanding the range of adapters for industrial protocols such as Allen Bradley, BACNet, Beckhoff TwinCat ADS, and Modbus. These new adapters also support advanced configuration options at both protocol and transport levels. ?? New Data Processors and Sinks Version 0.95.0 introduces several new pipeline elements, including a Datetime from String processor and a Microsoft Teams sink. ?? Introducing the Go Client A notable new feature is the Go client for Apache StreamPipes, available as an early preview. This client enables interaction with the StreamPipes API from Go applications. We are actively working to expand the feature set supported by the Go client. ?? Data Explorer Enhancements We've harmonized the technical framework of all data explorer widgets in recent releases. StreamPipes 0.95.0 now utilizes the powerful Apache ECharts library for rendering widgets, enhancing the look and feel while laying the groundwork for feature extensions such as advanced configuration options for data explorer widgets, including axis configuration and data zoom. ?? Improved User Experience Nearly every aspect of the StreamPipes user interface has been refined. Modifying existing adapters is now more intuitive, several bugs in the pipeline editor have been resolved, and the installation process for extensions has been enhanced for better usability and performance. The Apache StreamPipes community has already begun working on the next release, 0.97.0, since April. Stay tuned for exciting new features that we'll be announcing soon! ?? And as usual, the new release is available for download on our website, and also find the blog post with the most important feature highlights! The Apache Software Foundation #IIoT #OpenSource #Analytics

Mobile alerting for Apache StreamPipes. SIGNL4 extends your IoT scenarios with alerts via push, SMS, email and calls. Including acknowledgements for optimized transparency and shift planning to alert the right people at the right time: https://lnkd.in/ea-HFVDA #Modbus #OPC-UA #MQTT #Alerting #Cloud #SaaS #Technology #IoT #IncidentManagement #IncidentResponse

StreamPipes is part of the OSPP 2024 ???? We are pleased to announce that we are participating in this year's OSPP Challenge for the second year in a row. The Open Source Promotion Plan (OSPP) is a summer program organized by the Institute of Software Chinese Academy of Sciences and aims to encourage college students to actively participate in the maintenance and development of open source software. We've submitted two exciting projects and can't wait for the students to get involved: 1?? Enriching Go Client Features For the last few months we have been working on a client that supports interaction with StreamPipes via Golang. Within this OSPP project we would like to enrich the functionality, implement an end-to-end testing framework and improve the documentation. 2?? Integrating Apache IoTDB with Apache StreamPipes StreamPipes already provides options to read from and write to an Apache IoTDB. In this project, we want to explore a deeper and more advanced integration of StreamPipes with IoTDB, e.g. taking into account #TSFiles. After a successful collaboration last year, we're excited to see the results of this year's OSPP submissions. Interested in participating in our OSPP challenges? Check out the project details and application information in the comments below! ?? #ospp #opensource #community #studentengagement #golang #iotdb #iiot

?? New Blog Post Alert! ?? Our latest article explores practical applications of StreamPipes in machine learning workflows, focusing on seamless IIoT data extraction and model interoperability via ONNX. Discover how StreamPipes simplifies IIoT data extraction with our Python client, and learn how to apply your ML model to streaming data effortlessly. Join us as we delve into an anomaly detection scenario: detecting leakages in a water pipe system. Read more: https://lnkd.in/eG9sr9h2 #ML #ONNX #AnomalyDetection #IIoT