Testing applications against Hadoop distributions is not fun, either for application developers or end users, and it takes up too much precious time.
According to Alan Gates, co-founder of Hortonworks and ODPi member, that’s the issue the Open Data Platform initiative (ODPi) is here to solve: create a single test specification that works across all Hadoop distributions so developers can get back to creating innovative applications and end users can get back to making money, or curing cancer, or sending people into space.
“That’s where ODPi sees itself bringing value,” Gates said. “Specifying not what’s in this software, or writing competitive software, but specifying how this software is installed where it can be used regardless of which distribution it is, how is it configured — all of those questions, which maybe aren’t as exciting as developing new software, but they’re questions you have to answer well in order for people to use your code.”
Gates gave a keynote session about the nonprofit organization at Apache Big Data in Vancouver in May. ODPi, which now has 29 member companies and 35 maintainers, released its first runtime specification on March 31 of this year.
It’s also working on an operations specification around using Apache Ambari that is slated to launch July of this year.
“Those specifications are frankly a little boring,” Gates said. “It’s just [saying things like]: here is how to lay out the directory so people can find the config files; here’s the environment variables that must be set so people know where you put the binaries; don’t move the binaries around on people and don’t take some away; don’t change public APIs; don’t rename .JARs.
“None of this is rocket science — despite the little rocket in our logo — but it’s all very necessary,” Gates said.
Gates sees three main constituencies for whom the ODPi is trying to make the Hadoop ecosystem a better place:
-
End users: “We want [end users] to be able to run ODPi-compliant distributions with ODPi compliant applications on top and be able to mix and match and not worry about who they bought which piece from.
-
Application developers and ISVs: “We want them to ‘test once, run anywhere’ and reduce the cost of building the applications. The more applications they build, the faster the ecosystem grows and everyone is happier.”
-
Distribution providers: “We want to give them guidelines on how to install and set up their software so the two groups above get their benefits.”
Gates said the ODPi doesn’t write much code, but any code it does write is contributed back to the Apache foundation’s projects. Ambari and Bigtop have see the most commits from ODPi, he said.
“We are very committed to making sure all that work that we do feeds back up into the Apache communities and is used by them,” Gates said.
Watch the complete video below.
