Hedron: Or my dissatisfaction with local development.

I like things that work. I think most technicians do, but as a web developer I have a very serious problem. My most effective environment for doing web development is the one that exists on my own personal box. It can also be a rather impractical place to develop because most of my customers (current and historic) are on rather customized server stacks. Typically, the host has customized the environment to their own specifications. It's not uncommon to find additional services like solr or maybe a memcache server in the mix. I work for a company who builds its own platform but I personally have no control over what environment gets built for a customer, nor do I have any influence over where the tech stack is going at a higher level.

I'm not an Ops guy. I don't like screwing with server configuration. It makes me cranky. But I'm also not blind and I do like artifact generation. I have a long history with what we in the Drupal community think of as "artifacts". We us a tool called drush make and build and distribute Drupal profiles (i.e. artifacts) through it. In recent years, PHP itself has adopted this same style of approach with composer.json/lock files as have many other programming communities. If you look at what Docker does with their docker files and things like docker-compose, these are tools that build consistent "ops artifacts"... regardless of where they're built (ish). Following these threads of commonality, I decided I wanted to build a better local development environment tool. Hopefully one that would be inclusive in nature and give communities outside the ones I've historically associated myself with an opportunity to benefit and contribute.

Hedron:

Hedron is an PHP and Docker-based tool for generating ops and development artifacts that work in tandem to give you a local development environment with the flexibility to match what exists in your hosting environment. For ops, Hedron depends on docker-compose to build out clusters of containers providing the same basic environment as your production system. For development, Hedron embeds a Drupal 8-inspired plugin system to give developers the flexibility to build their development artifacts in whatever way makes the most sense for them. I've personally spent some time on a Drupal 8 workflow because I know Drupal 8, and it works well enough for people to experiment with and begin improving.

Hedron leverages many of the tools Drupal and PHP developers are already used to. Out of the box you'll find:

  • Familiar Symfony components
  • A Drupal 8-inspired plugin system rewritten from scratch
  • A powerful set of tools for responding to git push commands
  • On demand development environments per git branch
  • Helpful CLI tool
  • Room to contribute ;-)

Hedron is broken up into two separate components today with Hedron and HedronCLI. HedronCLI is designed to ease using and installing Hedron with built in commands for installing the core of Hedron, adding clients and projects, and managing docker clusters per project with ease. In addition to this my father has helped me put together an initial docker-compose build that should work well for Drupal 8, and we've made that available along side Hedron on github. Our main website should have all the instructions you need to get started.

What you'll need:

Hedron has a few dependencies:

  • php 7.x
  • composer
  • docker (and docker-compose if they're not packaged together for you)
  • git

You'll need these 4 things installed on your local machine. Ultimately I'd like to see Hedron evolve to a point where it ONLY needs docker, but I'm not certain how best to do that today, and I'm more interested in forward movement. (Fail fast, fail early)

Caveats:

Docker-for-mac is SLOOOOOW. I mean like, really slow. This is due to an issue with OSXFS shares. There are work-arounds, I've not yet delved into what's involved, so just keep in mind if you're on a mac, YMMV a lot depending on the newness of your machine and as always "patches welcome".

I hope people give Hedron a try. I've put a bit of thought into how I personally want to develop going forward, and I feel like this is moving in the right direction. I think there's a ton of potential for future improvement and expansions into other offerings based on Hedron. Give it a whirl and let me know what you think.

13 January 2017

I'm definitely aware of DrupalVM. Jeff's also familiar with what I'm doing at least in passing, and there are definitely some differences. While DrupalVM has settings to give you the features you need to develop, Hedron gives you control over the actual containers. This means that if you're using docker at the hosting layer as well, you could have identical environments locally and in production. I just feel like that's a benefit that goes two ways. For Developers, they get an environment they know matches production, and for Ops, they will spend less time playing whack-a-mole with bugs that only manifest in production. We could argue whether anyone is really doing much of that, but I think it's just a matter of time before Docker is the "lingua franca" of ops work, and I think there's a big opportunity for the same sort of transformation in the development environment.

Ultimately, while both tools help the developer, Hedron wants to do other things besides just single VMs and besides just Drupal. Drupal's the only plugin I've built for it today because I know and understand Drupal, but Wordpress, Magento, ruby, python, nodejs and more could all be built from a Hedron workflow, and I think that's a pretty big differentiator.

13 January 2017

Kris and I walked through an early demo of Hedron, and I like what I see—one quick note on the 'ops artifact' idea, though... if you're deploying your code to any of the typical hosting providers (e.g. almost any in existence today), you likely don't have control over your infrastructure to the point you could run your own Docker containers on it.

Things may change as the hosting industry continues to evolve, but one reason I think Drupal VM will still get a lot of traction in the near term is that it can flex to suit almost any kind of infrastructure and deploy style (and it can also be used to manage production servers!).

For Hedron, one thing that I really would like to see is the ability for hosting providers to actually support custom Docker containers (like those you'd generate in Hedron) so you could match your development and production environments _exactly_ (minus any helper utilities like Xdebug if you have a secondary container running it).

13 January 2017

I whole-heartedly agree. I would love to see Hedron progress to the point where it could be involved at the hosting layer so that you have the exact same software powering your local development tools and your production hosting. Hedron is in some ways my anticipation of that potential future. Whether it happens or not depends on how many people want to help make that a reality. I hope I'm not alone in that. :-)

David Hernandez (not verified)

13 January 2017

Have you tried Docksal (drude) ? https://github.com/docksal/docksal

It looks to be very similar to what you are working on. Docksal has been in development for quite a while and is fairly stable at this point. It's been made agnostic, so we have projects with D7/8, Wordpress, straight php, flat html, etc. We're also using it for CI. 

13 January 2017

I'm aware of Docksal but it came on my radar pretty late in my work on Hedron. I chose not to look at it just from an "influence" perspective, but my arch is pretty well in place now so I'll definitely browse what they're doing.

Dave Anderson (not verified)

19 January 2017

Have you looked at Kalabox? http://www.kalabox.io

It's very similar to what you're working on and is supported by Pantheon.

Lucas (not verified)

24 January 2017

Have you looked at Docker4Drupal? Just one more alternative as it looks like.

It's good to see the community building things to ease the development process, however I'm afraid of these many "alternatives" to solve the same problem. Most are not even well established like Drush on CLI for example. Maybe sometime soon it will be nice if guys behind Docker4Drupal, Docksal, Hedron and just another tool a few colleagues are about to release publicly (which does pretty much the same thing) should talk and come into a consensus and maybe one or another abandon a certain project in favor of a a greater one for the whole community. 

Many options to chose from are sometimes bad... we've seen this happening in the frontend community with their huge number of micro-frameworks to do pretty much the same thing. 

Gary Mort (not verified)

21 February 2018

I'm all for many options at this point.   The Docker eco-system is completely geared for Production usage, not local Developer usage.   A docker system is geared to run A single website using many micro-services for security and scalability, not 4 website projects on your local machine at the same time.  And there are not many other options for a docker like tool other then Docker.   I see Docker as stagnating in many areas since the focus is on production.

Many options creates competition to be the best - or in some cases to be the best for that specific purpose.  Create many options, devs will initially pick the ones they like the best and that will thin out the herd.

 

24 January 2017

In addition to the others that people have listed there's also Kalabox which is pretty awesome for its push/pull commands. I found half a dozen others too when I did some research on this a year ago. 

David Snopek (not verified)

24 February 2017

I've also got a Docker-based Drupal development environment thing :-)

We should try and do a BoF at Drupalcon to see what opportunities there are for collaboration!

Gary Mort (not verified)

21 February 2018

I have found a very affective solution is to use cputhrottle on hyperkit.

Hyperkit is the thing which connects everything from your Mac to the primary Linux VM from where every other docker image will run on.   As near as I can tell, the file sync process eats up a lot of CPU time randomly to do nothing.  IE a bunch of files change somewhere, anywhere, on your Mac[maybe the spotlight indexer just ran].  Hyperkit is notified of each and every change and has to go "nah, I don't care about that file".   While it does that, it jumps my CPU usage up to 400%[all FOUR CPU's running at 100% flat out].

Just install glances, brew install glances, and you can see the dang thing spiking.

You can get the tools you need from homebrew: brew install cputhrottle && brew install pidof

Then just grab my script from https://gist.github.com/gamort/1b9379ce8f30d07932be9a416cf17f55

Make it executable and run it.

You will have to run it every time you restart docker, it checks for and will throttle a specific process id - thanks to pidof.  When Docker restarts, a new copy of hyperkit is started.

The script is a little overly complicated.  I googled around till I found it and it is written to allow you to throttle many different programs and for different percentages.  The original one was configured to keep PhpStorm from going out of control.

So far 60% has worked well.  Bump it up if you run into problems.  I don't like to go above 100%

 

Add new comment

The content of this field is kept private and will not be shown publicly.