Java
fromMedium
15 hours agoSpark Internals: Understanding Tungsten (Part 1)
Apache Spark revolutionized big data processing but faces challenges due to JVM memory management and garbage collection issues.
Carnegie Mellon University has introduced Fray, a concurrency testing tool for JVM programs to catch bugs and replay them. Written in Kotlin and based on this research paper, Fray can't find all concurrency issues, but uses recent research to maximize the chances of detecting them. Fray uses shadow locking, a technique that incorporates extra locks to mediate access to shared resources in a specific order.