November resembles a final crescendo before the year draws to a close. Everything is still in full swing as we prepare to wrap up 2023. So, no wonder a bunch of things happened over the last 30 days. We launched the RisingWave Tutorials, a comprehensive guide that provides a deeper understanding of RisingWave and its application in real-time data processing. We also rolled out RisingWave v1.4 adding new upstream and downstream connectors as well as functions and operators for more complex data transformations. Last but not least, the roadmap for Q4 and beyond has been released, and we are grateful for the overwhelming feedback we received.
Don’t miss out on any of these — and more — by scrolling down. ??????
In November, we introduced several new features:
- Connection management. We introduced the connection management panel, a dedicated page to manage the lifecycle of all established connections. The supported connections now include both AWS PrivateLink and GCP Private Service Connect. With a unified conceptual framework, RisingWave Cloud allows one to easily set up a secure connection from RisingWave Cloud to an external VPC on either AWS or GCP.
- Microsoft login. We have added an additional login method: Microsoft login. Users can log in to RisingWave Cloud through their Microsoft account.
- Kafka sink enhancements. We have introduced several enhancements to our Kafka sink. A key new addition is support for the upsert + avro format. Upsert envelopes allow users to export DELETE and UPDATE changes, not limited to INSERTs. Using the more compact Avro binary format can also help save network bandwidth between Kafka and RisingWave.In practice, users often leverage Kafka sinks to incrementally export RisingWave materialized views into downstream databases, such as Oracle and MongoDB. Kafka serves as a fault-tolerant buffer that can handle temporary downstream overload situations. Beyond Avro, upsert + JSON-with-schema is also supported in the v1.4 release.
- Recoverable CREATE MATERIALIZED VIEW. Before recoverable CREATE MATERIALIZED VIEW was introduced, failure would require restarting the entire time-consuming backfill process from the very beginning. Now we persist backfill progress periodically, so the process can be resumed from the last checkpoint after a failure.
- Check out our Upgrade RisingWave in a Kubernetes cluster guide to learn how to upgrade RisingWave with Helm or the Kubernetes Operator.
- Explore how to establish a secure and direct connection between RisingWave Cloud and a source or sink that is located within a virtual private cloud. See our VPC connection page for more details.
- The JSON functions and operators page sees a multitude of new functions and operators. Look out for the following if you are interested in manipulating JSON data in RisingWave.- New JSON operators: ?@>,?<@,??,??|?, ?&,?#>, #>>,?-?and?#-.- New JSON functions: ?jsonb_extract_path,?jsonb_object,?jsonb_pretty, and?jsonb_strip_nulls.
- Use the SHOW PROCESSLIST command to see the workload of the system. For more information, see the documentation.
- Add comments to your tables and columns in RisingWave with the COMMENT ON command. For more details on the syntax, see the documentation.
- RisingWave now supports sinking data to Redis and Google BigQuery. Check out our Redis and BigQuery guides to learn more about the process
- Learn how to connect Looker, a business intelligence platform, to RisingWave, by following along with our new Looker guide.
Two months ago, we hit the 1000-member mark, and our Slack community keeps on growing! It is a great place to learn about Rising Wave and connect with other data practitioners. Whether you have run into a problem deploying and using RisingWave, or have a question to ask/want to offer help, we'd love to have you on board. Let's create together a space where we can learn from one another while continuing to improve RisingWave. Join today!
