Synnefo Architecture

Hello everybody,

On a previous post, almost a year ago, we described the services and REST APIs of Synnefo.

Today, we have 2 new diagrams that show Synnefo's layered approach and overall architecture in more detail. First, we extended the diagram from the previous post that presented the zoom-out view, to include all layers from the hypervisor up to the clients (Web UI or command line). We also updated the API layer to reflect the complete OpenStack compatibility that has been included in the latest version. We have now fully integrated Neutron, Cinder and Keystone along with the previously implemented Nova, Glance and Swift:

The second diagram is created from scratch and zooms in to unveil the whole internal Synnefo architecture, showing all Synnefo components and the interconnections between them:

Note that in the left part (Compute/Network) the cloud layer (Synnefo) interacts with the cluster layer (Ganeti clusters) only via HTTP in the effect path and only via AMQP in the update path. The cluster layer does not access any DB, allowing us to scale linearly by adding new nodes on a Ganeti cluster or new Ganeti clusters altogether, even in geographically distinct locations.

Also note, that with Archipelago one can operate over multiple storage technologies
transparently, without the need to change anything on the upper layers, may these be Ganeti or Pithos.

Just to give you a preview on what's coming next, along with many new features, starting v0.16, Pithos will become a thin layer on top of Archipelago or in other words Archipelago will become the backend for Pithos. This will allow for 2 things:

• Complete unification of cloud storage as seen by the upper layers
• Complete abstraction of the actual data store underneath

So, hope you find the above interesting and hope the diagrams help everybody to understand the Synnefo architecture a little better.

Thanks,
Constantinos

Synnefo @ FOSDEM 2014

Hello everybody,

the schedule for the Virtualization and IaaS devroom of FOSDEM 2014 has been announced and we are more than happy to be a part of it. After last year's introduction, this year we will be talking about advanced storage integration on Synnefo with Archipelago and Ceph. Check out the talk abstract here.

Of course, we will be making a quick intro to the stack for those who haven't heard about it before, and also do a live demo.

So, join us at the talk, we'll be happy to meet you.

See you all in Brussels.

Synnefo v0.14.10 Released

Hello everybody,

we are pleased to announce that today we released Synnefo v0.14.10.

You can find the Debian packages on our apt repository (apt.dev.grnet.gr) under wheezy.

You can also check out the upgrade notes here:
http://www.synnefo.org/docs/synnefo/latest/upgrade/upgrade-0.14.10.html

As you may already know from a previous email on the list:
https://groups.google.com/forum/#!topic/synnefo/rjUMyZTJmWU

Synnefo v0.14.10 is the second transitional package that will help you to smoothly upgrade to Debian Wheezy. Synnefo v0.14.0 is not compatible with Ganeti 2.6 any more, so you will also need Ganeti 2.8 to proceed. You can find the corresponding Ganeti package (snf-ganeti) on our apt repository too. The version you need is:

2.8.2+snapshot1+b64v1+hotplug3+ippoolfix+rapifix+netxen+lockfix2-1~wheezy

The patches: hotplug3, ippoolfix, rapifix and netxen are already merged in the official upstream and will be part of Ganeti 2.10. So, we just backported them in this package. The patches: b64v1, lockfix2 are two small patches that fix some minor issues. The patch: snapshot1 implements some new functionality regarding snapshots which is not used currently by Synnefo v0.14.0, so the codepath is actually inactive, but is there to get stress tested in our production deployment.

So, go ahead, give them a try and please report back any problems or bugs you may find.

Enjoy,
from behalf of the Synnefo team,
Constantinos

Archipelago: officially open source

Hello everybody,

We are pleased to announce that today we are releasing our custom storage layer, Archipelago, as open source software under a 2-clause BSD license. Archipelago has been running in production for over half a year without problems, so after a substantial cleanup we decided to open it up to the public.

Archipelago is a new storage layer that decouples Volume and File operations/logic from the actual underlying storage technology, used to store data. It provides a unified way to provision, handle and present Volumes and Files independently of the storage backend. It also implements thin clones, snapshots, and deduplication, and has pluggable drivers for different backend storage technologies. It was primarily designed to solve problems that arise on large scale cloud environments. Archipelago is written in C.

Please check out the official documentation:

http://www.synnefo.org/docs/archipelago/latest 

You can try it out for yourself, following the instructions found here:

http://www.synnefo.org/docs/archipelago/latest/archipelago_deploy.html

You can also find the code here:
https://code.grnet.gr/git/archipelago

Debian packages can be found on our apt repository under unstable. Add the following to your sources to use them:

deb http://apt.dev.grnet.gr unstable/

deb-src http://apt.dev.grnet.gr unstable/

Finally, on the apt repository, we provide an Archipelago ExtStorage provider, for those that want to use Archipelago with Ganeti.

We hope Archipelago is going to prove useful to others as it did to us.

As always, for comments, questions, or bug reports feel free to contact us at:
synnefo@googlegroups.com

Enjoy,

the Synnefo team

Synnefo @ USENIX ;login: (Oct '13 issue)

Hello everybody,

The October issue of USENIX ;login: is out!

And it features an article we've written about Synnefo. Good news is that it is open for everybody, not only USENIX members. So, go ahead and check it out. It is entitled:

"Synnefo: A Complete Cloud Stack over Ganeti"


Enjoy,
Constantinos

Synnefo @ Ceph Day London

Hello everybody,

The schedule for Ceph Day London has just been announced!

What's more, we will be presenting Synnefo at the event showing how we manage to unify cloud storage (Files, Images, VM disks) using

Synnefo + Ganeti + Archipelago + RADOS in production.

We are also really excited to meet the guys from Inktank and other Ceph users, learn and exchange opinions.

So, if you are in London and the above sound exciting, please join us at the event.

See you in London.

A great GanetiCon 2013 has ended

Hello everybody,

as you already know, the first Ganeti conference, GanetiCon 2013, took place last week in Athens. We were extremely happy to be part of it, not only during the organizational procedure, but most importantly during the conference itself, participating in the various design discussions and user reports.

It was great meeting most people from the Ganeti team and other Ganeti users from Greece and abroad. We also had double the pleasure to meet in person happy users of Synnefo, too.

The conference went smoothly from beginning to end and everybody seems to have enjoyed the event. The venue, food and schedule were great, since everyone tried their best and that really showed.

All in all it was a wonderful 3-day event.

For all of you that couldn't make it, please take a look at the updated GanetiCon site for photos and presentation slides.

Let's hope we made a good start for Ganeti this year, and more GanetiCons are ahead the next years to come.

See you all at the next GanetiCon,
Constantinos


P.S.: a special thanks to the guys from Inktank and especially Patrick for sending us all the great Ceph swag, even though they couldn't make it to the event.