Microsoft has announced the preview of Azure HDInsight 3.6 with Apache Spark 2.1.
“We are pleased to announce a preview of Azure HDInsight 3.6. We are enabling this preview to get feedback on Apache Spark 2.1. You can try out all the features available in the open source release of Apache Spark 2.1, along with the rich experience of using notebooks on Azure HDInsight.”
The open source Apache Spark 2.1 release brings several improvements for developers. These improvements range from Structured Streaming to Apache Kafka (version 0.10) with Spark Streaming.
For more information on improvements in Apache Spark 2.1,
check here.