I wrote a book for O'Reilly on scaling machine learning with Spark specifically. My second book is coming out on how to improve high-performance Spark, the second edition. Started my career in the machine learning space 15 years ago, moved into data infrastructure, batch processing, and a year and a half ago I moved into the data streaming space, which I think it's what's going to help us pave the future in the data.
As companies race to scale revenue operations efficiently, a new role is emerging at the intersection of technology and business strategy: the GTM (go-to-market) engineer. According to ZoomInfo data, hiring for the role has doubled year over year for the last two years, with hiring peaks in January and July. In mid-2025 LinkedIn had over 1,400 postings for GTM engineering roles, with over 3,000 listed in January of 2026
In 2025, AI became officially unavoidable: It had been lurking in the background before, as early adapters experimented with it. But this year, companies invested more than $202 billion in AI, a 75% increase from $114 billion invested in 2024. Major tech companies fought bitterly over AI talent, offering astronomical pay packages. There was a groundswell of demand for talent, and unsurprisingly this spread to the demand for AI,
It is becoming increasingly difficult to separate the signal from the noise in the world of artificial intelligence. Every day brings a new benchmark, a new "state-of-the-art" model, or a new claim that yesterday's architecture is obsolete. For developers tasked with building their first AI application, particularly within a larger enterprise, the sheer volume of announcements creates a paralysis of choice.
1. Scala Coding's Fibonacci Check Question: Write a Scala program to find out if a given number belongs to the Fibonacci series. If the response is yes, return that number. If not, return -1. Solution: For n = 7 → Output: -1 (7 is not in the Fibonacci series) For n = 8 → Output: 8 (8 is in the Fibonacci series)
The combined company aims to deliver an open data infrastructure - unifying data movement, transformation, metadata and activation - while preserving freedom of choice for analytic compute and AI, according to the announcement. The company's vision for an open data infrastructure will reduce engineering complexity by automating data management end-to-end, and work across any compute engine, catalog, BI tool or AI model.
In any enterprise application, user-provided data is often messy and incomplete. A user might sign up with a "company name," but turning that raw string into a verified domain, enriched with key technical or business contacts, is a common and challenging data engineering problem. For many development teams, this challenge often begins as a seemingly simple request from sales or marketing. It quickly evolves from a one-off task into a recurring source of technical debt.
Muse helps creative and launch teams at Netflix understand which artwork and video assets resonate with audiences, and its growth required supporting advanced filtering and audience-affinity analysis at massive scale. To meet these demands, Netflix reports that it redesigned the data serving layer, cutting query latencies by about 50% while maintaining accuracy and responsiveness. Muse started as a Spark-powered dashboard backed by a modest Apache Druid cluster.
Lakeflow and OpenFlow reflect two philosophies: Databricks integrates data engineering into a Spark-native, open orchestration fabric, while Snowflake's OpenFlow offers declarative workflow control.
