OSX Yosemite + Ubuntu 14.04 / dual boot

Before to start playing with the commands suggested here let me clarify my context:

  • I had a Mac Book Pro with OSX Maverick with dual boot working with Ubuntu 14.04 through rEFInd boot loader.
  • Today I upgraded OSX to Yosemite and after everything was updated rEFInd stopped working.

So my goal was to make able to have rEFInd working again to switch between operating systems. My primary OS is Linux so if you are in a similar situation this may help:

On a Twitter search I found someone linked the following solution:

http://pastebin.com/n1MCYxsF

That worked for me, I got a few warnings and I did not need to run the ‘mount’ command. Good luck!

Looking forward Monkey v1.6 and Duda I/O v2

The next versions of Monkey HTTP Server and Duda I/O are very challenging, because both are very related and big changes are coming in terms of optimizations and general features.

As said many times, Monkey is the base HTTP stack for Duda I/O, for hence any improvement made on Monkey is inherit by Duda I/O, as well any extra feature is exposed too. Let’s see what is expected for the following major versions:

Monkey

The code base continue being optimized, we care a lot of performance on all environments, not just high-end production servers, also in Embedded based systems where we got a lot of traction.

The big news besides performance, is the new event loop (polling) mechanism implemented, so starting from v1.6 we support Linux and also OSX. Yes, I know…what about our speech about Monkey is just for Linux ?, here is the answer: As a HTTP server Monkey it self does not have too much things to offer on an operating system that is not widely deployed on Internet Servers, neither on Embedded systems, but we need to look beyond and think in use cases, the main reason of Monkey being supported on OSX is Duda I/O.

Duda I/O is a fast web services framework, in short words: it let you build scalable C web services with a very simple API. Nowadays OSX is a base platform for developers, so our move is to provide a stable and fully featured web services stack to the Apple world. Wouldn’t be great to write your scalable web services in C on your favorite OSX system and deployment later easily on Linux servers or devices ?, yes it is. This is very exciting for us and continue involving a lot of work, but fun :) .

Duda I/O

When I mention a web services stack I refer to a professional product, fully documented and that have been used by production systems on the last two years. So major versions do not happen too frequently, we focus on long term support and stability.

The next version DST-2 (Duda Stable 2) will be released with great changes that covers a bit of everything:

  • Co-routines support (dthreads)
  • New routing model, do you like URLs like /users/:id/about/:phone/ ?, there you go.
  • Re-designed Console and optional Dashboard UI
  • Streams
  • Docker Support
  • More Documentation
  • Duda QA: Quality Assurance.

I mentioned Duda QA, so let me explain. Starting from DST-2 there is a new project that is part of Duda ecosystem, it deliver a complete Test system for Duda. For every Object API exists an unit test, as many changes are underway we need to make sure to avoid regressions. That means that as of now there are around 200 tests under 1300 lines of code…and growing. So every time we do some changes, you as an end-user can trust that we are not breaking the API or anything similar. There is nothing more frustrating than a broken API/stack, so keep calm as we are really taking care of that. More details about the project and status here:

https://github.com/monkey/duda-qa

In general many things are coming, there are other features and improvements that I did not mention but I am sure I will once I take some time to write the Release Notes, by now the works continue.

Busy and Productive Weekend

Monkey

I planned to have a quite Saturday, but after check my email in the morning, I found a possible security issue reported for Monkey (1.5 onwards), it may lead to a DDoS based on the new optional FDT feature, so it was time to research and release a fix as soon as possible.

So Monkey v1.5.3 was released which include the fix for the problem reported, i also take the opportunity to include other patchs sent by the reporter, so the goal was accomplished: act transparent and quick.

Duda I/O

Also during the week i have been working in a new look & feel for Duda I/O project, you cannot imagine how hard is to create an image for a Software that don’t have a UI (yet). So the project just got 3 major things:

New logo

Duda I/O is a Stack, for hence from my perspective I think this represents it nature and simplicity:

not sure if people would love it but I am sure it will fit very well on Project T-shirts and Stickers :)

New Web Site

As the new logo arrived, was a good time to complete the new responsive web site:

Disclaimer: I bought a bootstrap template and did the adjustments, i am not a designer.

compared to the previous project site this is a major upgrade.

Google Summer of Code

We are in the pencils down phase of the Google Summer of Code program, that means most of our students are doing their final adjustments on their projects as the final evaluations may start this evening, all of them have done a very good job, they deserve the credit. More news about cool stuff that was made will be public in the incoming days.

It’s a good jurney, life is moving a lot in a positive way, in all aspects…

Monkey & Yocto Project

Now it’s official, Monkey is a participant organization of Yocto Project.

Participate in Yocto Project is not just about software recipes, it’s about a compromise to contribute to make Embedded Linux environment compliant, and as an Open Source organization there is a huge responsibility into push forward stable and innovative technology, so for us this is really good news, make strong relationships is critical to align objectives and goals.

You can read more details about this in the Monkey Blog Post here.

RSS: http://blog.monkey-project.com/rss/

 

Writing Servers with focus and integration

I always believed that integration and focus are a success key for any kind of thing on this life. When talking about technology and servers specifically, integration may be a fundamental part to achieve different goals like good performance. Years ago was very valuable to have a software that can work on multiple Operating Systems, but there is a point that if you want to increase your performance limits you need to look forward for integration.

In the last years, Linux have become the best choice for Servers, it’s well known that every big company trust in Linux and also commit to their development: Redhat, Oracle, Google, HP, IBM and many many others. So we can say that Linux is doing pretty well on professional Hardware, but what about user-space software ?.

If we look in the HTTP Servers (or Web Servers) context, we will find that most used solutions are made to support different platforms and on specific build-time for a specific Operating System, different approaches are used to work better, in servers performance and resources consumption are critical, and when moving those solution to Embedded even more, so every taken decision may sacrifice something, we do in OS portability.

Monkey HTTP Server and Duda I/O are written with focus on the Linux Kernel, every Unix operating system despite their POSIX API, provides extra system calls that aims to extend functionality and offer features. Of course this kind of integration have a cost which is basically: do not be portable to other Operating System, so depending of your human resources you need to decide how to invest better: we do in focus and integration, and it works great.

As a reference i would like to mention which specific features we use and i hope people who write servers or any kind of networking application would take this in count. Of course  these are dependent of specific Linux Kernel versions:

Linux 3.14

  • TCP AutoCorking: let Kernel do smart buffering when sending outgoing data, the old fashion way is to do manual Cork over sockets (more).

Linux 3.9

  • SO_REUSEPORT: be able to reuse port-addresses, e.g: multiple threads under same process can bind the same port and let the Kernel distribute load across them (more).

Linux 3.7

  • TCP FastOpen: Reduce the number of round-trip required in a TCP conversation (more).

Also other very helpful interfaces exists such as accept4(2), sendfile(2), epoll(7) and splice(2), these are not the only ones but that should be considered when writing a Server for Linux.

We are in good times where performance matters and make a good design to make distribute load across multi-core architectures is fun and challenging. Again, if you write with focus and integration will be a win:win relationship between your network server and the HW behind it.

GIT blame me: Never miss EPOLLRDHUP again

since some time ago when using a specific Duda I/O feature to connect backend servers using a custom web service, some inconsistency existed if those backends closed the connection, for some reason this ended up into the web service missing the connection close event.

This problem was detected months ago but it was not easy to determinate the root cause, it affected MariaDB/MySQL package and custom web services that use the stack event loop to be notified on sockets connected to third party servers. Last night doing some example code for a different thing the problem could be reproduced, and the steps were very simple:

  • create a TCP socket
  • make it asynchronous
  • run a connect(2) to a specific backend
  • register the socket file descriptor into the events loop interface
  • shutdown the backend server

and when a connection close notification should arrive, that never happened. This ended up in an inconsistency of a zombie TCP socket without events notifications on it. Looking around and tracing Monkey HTTP Server found the following difference:

  • when registering a new socket in the events interface, the following flags are always associated to a socket: EPOLLERR, EPOLLHUP and EPOLLRDHUP.
  • when someone change the socket direction (read to write, write to read), all flags are overwritten but on this time just EPOLLERR and EPOLLHUP were used, EPOLLRDHUP was missing!

Googling around i found this relevant comment:

A socket listening for epoll events will typically receive an EPOLLRDHUP (in addition to EPOLLIN) event flag upon the remote peer calling close or shutdown(SHUT_WR). This does not neccessarily mean the socket is dead. Subsequent calls to recv() will return any unread data on the socket and eventually “0” will be returned to indicate EOF. It may even be possible to send data back if the remote peer only did a half-close of its socket.

The one notable exception is if the remote peer is using the SO_LINGER option enabled on its socket with a linger value of “0”. The result of closing such a socket may result in a TCP RST getting sent instead of a FIN. From what I’ve read, a connection reset event will generate either a EPOLLHUP or EPOLLERR. (I haven’t had time to confirm, but it makes sense).

There is some documentation to suggest there are older Linux implementations that don’t support EPOLLRDHUP, as such EPOLLHUP gets generated instead.

After apply the following fix, the issue have gone:

https://github.com/monkey/monkey/commit/2d5e48d7697dd6b1ea3aaed48406d74e5d3ab097

Now wondering when (and who) introduced the problem:

$ git blame src/mk_epoll.c -L362,+1
  10c3eb27 (Eduardo Silva 2011-09-15...) event.events = EPOLLERR | EPOLLHUP;

 

Git blame me and lesson learned… never miss EPOLLRDHUP again.

Note: the fix have been backported to Duda-Stable-Branch 1 (DST-1), so people developing services only need to update their stable stack version.

Duda I/O is an Enterprise class stack

Duda I/O web services framework project was started almost two years ago, initially as a Monkey HTTP Server extension to create scalable web services in C, nowadays both components together are a complete stack to develop any kind of event-driven web backend application that relies on HTTP or Websockets.  As mentioned in early post, the web is not just about images, CSS or any JS scripts, its about also about fast data transfer and scalable services that aim to optimize resources usage with low overhead.

Sincerely i would say that Duda I/O and Monkey projects are the Mozilla of the backend side, of course we are far of their commitment but i would say the mission is the same: promote openness, innovation & opportunity on the Web.

Scale a C project at a community level is hard, due to the target market, the number of early adopters is very low compared to any  GUI or Web (front end) project, but once it get’s in and it works.. things changes in a positive way. At the moment there are a couple of Duda I/O instances running in production in private companies from US, Sweden and Germany, and i can conclude by their feedback the reasons of the adoption are pretty much the same:

  • High performance: all implemented web services requires some kind of real-time processing data coming from HTTP requests.
  • Open source stack: proof of good security handling, no back-doors. What really matters is that “It is open” (not talking about money).
  • Proof of years of development and commitment, the project will not disappear because of a one man project.
  • Good support: on early stages of evaluating the stack, their always receive good support through the mailing list or private emails.

Also much of the latest changes on the stack, comes from end-users feedback on topics such as documentation, development environment, API changes, etc. So this year have been a big step forward on terms of adoption and features improvements, next year will be even more fun and of course more hard work on:

  • Native deployment, native packaging for different Linux distributions.
  • Co-routines support to make easier to defer async work from callbacks.
  • SPDY & HTTP/2.0 .
  • New QA environment, stability and avoid regressions are high priority.
  • Clustering mode.
  • Sandboxing.
  • …others.

Personally i am just happy with all goals achieved during this period and very thankful to be part of a small but great community.

Google Summer of Code 2013 and other news

This week the Google Summer of Code 2013 program has ended, and i am glad to see our Monkey Project students completed their projects successfully, for more details please check the official post:

 

monkeyblog_gsoc2013

Besides GSoC , a couple of more news:

There is also other things that will be announced shortly, stay tuned!