December 26, 2012

Book Review: Otherland Volumes 1-4

*warning: plot spoilers below*. Imagine yourself in the trenches of WWI. The next chapter you're in Ancient Egypt, then in Troy. A little while later you've gone through the land of the bugs, dark/twisted versions of the Worlds of Oz and Wonderland, and many more. Of course this is before you arrive at the heart of the universe itself, where the basic rules of reality no longer apply.
The nice thing about Tad William's Otherland series, which takes place in the Otherland virtual reality network, is that he takes great liberties in his artistic vision. Otherland doesn't conform to the rules of many other artificial realities portrayed in fiction that try to preserve a consistency in their environments, rather is was built by a secret cabal of the most powerful people in the world to serve their nefarious purposes, each of which has a completely different vision of their utopia.

At first I was a little apprehensive about the last bit, in general I'm not a big fan of stories with a huge super-secret organization that runs the show, but progressing through the series it began to grow on me. Most of the main protagonists in the story start off as simple peons in the big picture, stuck in the simulation network, unable to leave, narrating the chaotic events unfolding around them. As the story progresses, the roles of the protagonists and antagonists become more apparent, but these mostly arise as artifacts of their characters, it is just who they are and often the random luck of their circumstances that results in their impact on the story.

There are exceptions to this, one of the most interesting characters is the dark serial killer knows as Dread, who is employed by the leader of the Grail Brotherhood (those that built and are running the network) to act as an assassin in both the real and virtual worlds to meet their goals. Of course things aren't as simple as they seem as Dread has perverse ambitions of his own, and seeks to control the very thing which his masters built to rule the world. Furthermore Felix Jongleur, the worlds richest man and leader of the grail brotherhood, proves to be quite interesting as well. While a bit on the cliché side his ruthless quest for immortality at any cost sets up and drives the events of the series and as they unfold around him, whether to his liking or not, and serve as many of the focal points of the epic. If you do start the series, make sure to commit yourself to reading to the end (it consists of four rather lengthy novels) as while it starts off slow, the ultimate fate of these two characters alone are so vivid that I suspect they will linger in memory for a while.

At the heart of the series, the network, and namesake of the books is the Other itself, the semi-sentient AI serving as the Operating System driving the virtual reality. The story of the Other when fully revealed is a very sad one, and there is no happy conclusion for this being, though there is some vindication with the fate of Nemesis, an ALife entity which the Other creates to quash anomalies in the network. I feel that in itself is enough reason to read the series, while many of the protagonists manage to escape the network and achieve their goals, it's no happy-hollywood ending. Rather there are many deaths and fates far-far worse, and as events unfold no character manages to escape the network unscathed, having to bear their experiences for the rest of their life. For some characters its worse than others, those with abilities which permit them some control over the network quickly find those abilities turn on them and they become more of a curse than a blessing. It is a bit of a tale in irony as nothing is ever what it seems, and while Williams often reveals the basic facts that setup for the awesome conclusion as the reader goes along, it isn't until the very end that all the pieces fall into place and the final puzzle is solved.

The series is an epic in every sense of the word, and in some senses it can be seen as a meta-epic, as other previously established epics such as the Odyssey and the Lord of the Rings are folded into the mix as characters go through those worlds, though often with a dark twist. I very much recommend the series to any sci-fi fan, the last book is so compelling I could not put it down (read the 900 page novel in a little under a week!) and I consider it to be one of my top three favourites of all time.

Until next time, happy dreams!

read more

December 19, 2012

isitfedoraruby updates

As a follow up to the GSOC, I've been mentoring several talented high school students from around the world who have been adding many cool new features and functionality to the isitfedoraruby webapp as part of the GCI contest. They have been doing a phenominal job on the site and an update sharing their work is long over due!

  • Nico (from Argentina) has deployed the site to openshift, wrote scripts to syncronize the db on a daily basis, automated the production push process (whenever we merge our commits into the 'stable' branch on github, the site automatically gets updated), is working on i18n internationalization support, and much more
  • Kendhia (from Algeria) has been working on features involving 'gamification', eg importing the contributors that maintain fedora packages, and displaying that data in visually appealing manners (including badges!)
  • Mark (from Honolulu, Hawaii) has been working on adding more visualizations to the site, most notably graphically representing the packages a contributor maintains using d3.js here. Mark also designed a logo for the fedora-ruby sig, which can be seen above.
  • Animesh (from New Delhi, India) implemented a timeline of contributions, so we can see a summary of all updates, both on the rubygems side and those in Fedora. He also wrote up and made a screencast of a great guide on how to get a simple hello world rails app up and running on the Fedora/Ruby stack.

We aim to keep driving the site forward, now that we've automated the push and syncronization process, getting new features into production should be very trivial. I'd like to thank the talented students for all their hard work and really look forward to continuing to work with them in the future!

read more

November 19, 2012

Fedora joins Google Code-In 2012

Google Code-In is the pre-college program for 13 to 17 year old students to earn prizes for completing tasks in various open source projects.,

This year will be the first year that the Fedora Project participates. We’re looking to build on many years success with Google Summer of Code, but the stakes here are higher. Unlike college students, who learn real life lessons the hard way, pedagogically you can’t allow high school students to truly fail. But failure is built in to the open source ethos. A fine line must be trod.

If you are a 13 to 17 year old or know one who is interested in code, design, documentation, just about aspect of an open source project, have them check out what Fedora has as tasks. They should also look at the wider program. Students can do work across multiple projects, or dig in deeply to a single project. Ideally, tasks are designed to be done within a few hours or days.

If you have any questions, I encourage you to contact Buddhike Kurera, who was last year’s GSoC admin. IMO, he has been doing a fabulous job driving the Fedora Summer Coding group. Joining the Google Code-In is the next step in his vision of how Fedora can work with students in these sort of contest/internship-style programs. I’m excited and curious to see how it all goes.

MWikipedia: M (named em ) is the thirteenth letter of the ISO basic Latin alphabet.

October 22, 2012

Google Summer of Code Mentors Summit

Friday I flew out to Mountain View, CA to attend the Google Summer of Code Mentors Summit hosted at the Googleplex, Google's worldwide headquarters. Every year, two mentors from every organization are sponsored by google to attend the summit and I was offered the extra slot by Buddhike, the Fedora summer of code admin.

After the inevitable hubub w/ delayed flights and such (surprisingly my stow-away bag actually made it!), I arrived at the hotel on Friday night intime for the pre-summit dinner. On Saturday shuttles were available to transport us from the hotel to the Googleplex which I have to say in quite impressive. From architecture of the buildings, to the modern artwork scattered around the complex, to the meals/beverages/snacks, and even random bicyles available for any employee to just pickup and use, the complex lives up to the legend! (hover over images for descriptions, click for full sizes)

googleplex1googleplex2

The summit was very much developer centric which I very much enjoyed, as most of the other conferences I attended included a mix of sys-admin and developer tracks. Obviously the driving event was a summer of _code_, so many of the attendees were software engineers themselves, mentoring students doing the development. Some projects were bigger than others, some well established, some not, but it was a great mix of contributors from across the industry working on many cool things.

roundtable1roundtable2

Among the best of the presentations I attended included roundtable discussions on the latest languages and language features being used in the industry, different collaboration tools that the projects are using and their thoughts on them, and one on developing tools to quantify and measure community metrics and individual contributions. During the lightning talks (one minute a piece), I showed off Zuhao's isitfedoraruby project, and extensively promoted his work, as well as Sammy's and Nitesh's contributions to Aeolus throughout the entire conference.

googleplex3

I also held an impromptu session on the JSON-RPC protocol during the afternoon which was a last minute thing, but was able to leverage my presentation and demo of rjr in Brno from a few months back. This led to an good discussion with some interesting ideas emerging from it and even a couple adopters!

aeolus stickers on the swag tablejsonrpc at the gsoc

Saturday night was the Summit Dinner Party at the hotel, italian food, good mingling, and even a dip in the pool and hottub! This morning I checked out of the hotel and attended more sessions before wrapping up the trip (I am currently blogging this from SFO prior to catching my red-eye home).

the rockies 1the rockies 2

I have to say this might just be the best conference I've been to so far. The talks and presenters were phenomenal. Even the attendees were constituted of some of the brightest of the industry, so many conversations lead to insightful perspectives and different conclusions than I hadn't come upon so far. I hope to be able to participate in the GSoC and maybe attend the summit again next year. Until next time, happy coding!

read more

October 03, 2012

Reactor Pattern + Thread Pool = Win

RJR uses eventmachine on the backend to process incoming messages over tcp/amqp/http/websockets/etc and dispatch json-rpc requests to registered handlers. It also employs a built in thread pool to hand off requests to the dispatcher so that the reactor isn't blocked. What results from combining the reactor pattern with a thread pool is a highly reliable concurrent event processing system.

The reactor central to eventmachine and the design pattern itself processes events one after another in a serial manner. Events are executed in the order that they arrive and no event executes until the one before it completes. Events can come from multiple sources and the reactor will take care of the serialization necessary to ensure data integrity.

A developer initializes the necessary connections and receives events by registering callbacks to be triggered on their invocation. Since the reactor blocks on any operation, the developer needs to ensure that his/her callbacks execute quickly and return control back to the reactor to continue processing events. Threads can be used to accomplish this, but spinning up a thread is an intensive operation, so a managed thread pool can be used as a nice solution that allows the developer to execute callbacks quickly and asyncronously without blocking the reactor.

Of course any resources leveraged by the callbacks will need to be protected from concurrent access but the nice thing is that if care is taken, the reactor itself can be used to handle this. In the callbacks launched by the thread pool, the developer can leverage the event machine reactor to schedule additional work that will be executed serially with the rest of the work already there. What results is an elegant / simple solution to schedule and process work concurrently with a built in mechanism to protect shared resources. Optimally the thread pool will incorporate a timeout mechanism that is able to kill jobs that exceed the timeout and restart their worker threads to keep things moving smoothly (as RJR does).

I would give this combined design pattern a name (perhaps Reactor Pool?) but I'm sure someone out there probably has already thought about this... and blogged about it... and uploaded an article to wikipedia... etc...

read more

September 27, 2012

Aeolus at the Ohio LinuxFest - Part IV Heading Out

Be sure to see the first three parts in this series which contains the complete overview of my demo for the conference. I also have a minimal set of slides which I've uploaded here (original showoff source and commands).

And thats about it! I head out in a short bit for my 7hr drive there (stopping in Buffalo) and back on Sunday. On Friday, I'm scheduled to present in the first slot at the 'Build A Cloud' Day, and then on saturday my OLF talk is at 3pm in room C110-112. Everyone who can make it is welcome to attend, and I'll try to take some photos and post from the conference but there is always a million things going on so no promises! :-)

Take care!

read more

September 26, 2012

Aeolus at the Ohio LinuxFest - Part III Conductor & CLI

See my first article in the series to set up cloud providers, and my second article on using the core Aeolus components.

Web interface

To setup the full Aeolus suite, simply:

  yum install aeolus-all
  aeolus-configure

Open a web browser and navigate to http://localhost/conductor to access the conductor web interface.

Login w/ the default credentials ('admin' / 'password').

To manage cloud provider, click 'admin' and 'providers':

Add a new provider and provider account, specifying the type and location
of the provider, access credentials, and uri of the deltacloud instance
through which to connect to it.

Manage cloud hardware and realms:

Manage pools of providers and environment which instances run in:

Manager users and rights:

Build images and start instances themselves on providers according
to any number of metrics:

CLI

To connect via the command line, setup a ~/.aeolus-cli file containing the credentials which to use to connect to conductor:

   :conductor:
     :url: https://localhost/conductor/api
     :username: admin
     :password: password

Yum install the command line utility:

   # yum install aeolus-cli

All operations available via the web interface are also available from the command line:

   $ aeolus --help
 
   $  aeolus hwp list
 
   $  aeolus hwp create --name demo
 
   $  aeolus image --help

</presentation>

That about wraps up the presentation content that I had prepared, besides a small / minimal set of slides that I will be posting shortly. I have also downloaded a few screencasts off our youtube channel as a backup plan incase things go awry with one of my machines (there is a nice / short one on the Audrey config server that I'll probably end up showing anyways to demo that component).

Stay tuned for my final post in this series!

read more

September 25, 2012

Aeolus at the Ohio LinuxFest - Part II Aeolus

To setup a few cloud providers to use against Aeolus, see my first blog post in this series.

deltacloud

To install, the core Aeolus component, deltacloud and the drivers for oVirt (RHEV) and OpenStack:

  # yum install deltacloud-core deltacloud-core-rhevm deltacloud-core-openstack

Deltacloud comes with a RESTful server, a command line client, and various programatic interfaces including C and Ruby. We'll start a few instances of deltacloud to serve requests to the various providers and connect to them via IRB / Ruby. A single deltacloud server can dispatch requests to as many cloud providers as desired, provided the correct HTTP headers are set such that deltacloud can resolve the remote provider entry point. But for this demo, to make it especially clear what is going on, two deltacloud instances are used:

  # API_PROVIDER='https://ovirt:8443/api'  /usr/bin/ruby /usr/bin/deltacloudd -i rhevm -e production --port 3004 --host localhost -u nobody -g nobody
  # API_PROVIDER='https://openstack:5000/v2.0'  /usr/bin/ruby /usr/bin/deltacloudd -i openstack  -e production --port 3005 --host localhost -u nobody -g nobody

Pop open an irb session and connect oVirt

  $ irb
  > require 'deltacloud'
  > c = DeltaCloud.new('admin@internal', 'cloudpass', 'http://localhost:3004/api')
  => <DeltaCloud::API:0x00000002d75968 @password="cloudpass", @username="admin@internal",
                                        @api_uri=#<URI::HTTP:0x00000002d75580 URL:http://localhost:3004/api>,
                                        @entry_points={:instances=>"http://localhost:3004/api/instances",
                                                       :images=>"http://localhost:3004/api/images",
                                                       :drivers=>"http://localhost:3004/api/drivers",
                                                       :instance_states=>"http://localhost:3004/api/instance_states",
                                                       :storage_volumes=>"http://localhost:3004/api/storage_volumes",
                                                       :realms=>"http://localhost:3004/api/realms",
                                                       :hardware_profiles=>"http://localhost:3004/api/hardware_profiles"},
                                        @features={:instances=>[:user_name, :user_data]},
                                        @verbose=false, @driver_name="rhevm", @api_version="1.0.0">
 
  > c.instances.collect { |i| [i.id, i.name] }
  => [["8b347be3-0a76-4c84-aa79-c017557540f5", "demo6"],        ["894fb094-c75d-49a8-b49d-5523346b46a3", "demo7"],
      ["98511559-46a3-4eb2-aba2-4c81c4e0d5b5", "i-1347974040"], ["29c4d6ae-87ab-46cd-8424-af05b96b1d91", "i-1348008963"],
      ["c1844032-45d4-4b17-9405-224b4942ace7", "i-1348425727"], ["12f8c82e-2837-49e0-8d41-745aa208021a", "i-1348516821"],
      ["9f3e115f-e097-4fdc-bfa3-5d634141c38a", "i-1348586396"], ["76b5bc43-6bc5-4c06-a92a-c86595bfc303", "inst1"]]
 
  > c.images.collect { |i| [i.id, i.name] }
  => [["00000000-0000-0000-0000-000000000000", "Blank"], ["d0a8df2e-1563-470e-bfd7-8384a400380f", "e3280dab-e719-4e6e-b954-55578acfe4b7"]]
 
  > i = c.create_instance("00000000-0000-0000-0000-000000000000")
  => <DeltaCloud::API::Stateful::Instance:0x000000025ddb60 @base_name="instance", @client=<DeltaCloud::API:0x00000002d75968 ...>,
                                                            @url="http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4",
                                                            @id="d637ee52-787d-48de-a6c4-a92b854677b4",
                                                            @objects=[{:method_name=>"stopped?", :type=>:state, :state=>"STOPPED"},
                                                                      {:method_name=>"running?", :type=>:state, :state=>"RUNNING"},
                                                                      {:method_name=>"pending?", :type=>:state, :state=>"PENDING"},
                                                                      {:method_name=>"shutting_down?", :type=>:state, :state=>"SHUTTING_DOWN"},
                                                                      {:type=>:text, :method_name=>"name", :value=>"i-1348597714"},
                                                                      {:type=>:text, :method_name=>"owner_id", :value=>"admin@internal"},
                                                                      {:type=>:link, :method_name=>"image", :id=>"00000000-0000-0000-0000-000000000000"},
                                                                      {:type=>:text, :method_name=>"image_id", :value=>"00000000-0000-0000-0000-000000000000"},
                                                                      {:type=>:link, :method_name=>"realm", :id=>"99408929-82cf-4dc7-a532-9d998063fa95"},
                                                                      {:type=>:text, :method_name=>"realm_id", :value=>"99408929-82cf-4dc7-a532-9d998063fa95"},
                                                                      {:type=>:text, :method_name=>"state", :value=>"STOPPED"},
                                                                      {:type=>:link, :method_name=>"hardware_profile", :id=>"DESKTOP"},
                                                                      {:type=>:text, :method_name=>"hardware_profile_id", :value=>"DESKTOP"},
                                                                      {:type=>:action_link, :method_name=>"start!",
                                                                       :id=>"d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :href=>"http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start",
                                                                       :rel=>"start", :method=>"post"},
                                                                      {:type=>:action_link, :method_name=>"destroy!",
                                                                       :id=>"d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :href=>"http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :rel=>"destroy", :method=>"delete"},
                                                                      {:type=>:action_link, :method_name=>"create_image!",
                                                                       :id=>"d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :href=>"http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :rel=>"create_image", :method=>"post"},
                                                                      {:type=>:collection, :method_name=>"public_addresses",
                                                                       :values=>[{:address=>"127.0.0.1", :type=>"vnc"}]},
                                                                                 {:type=>:collection, :method_name=>"private_addresses", :values=>[]},
                                                                                 {:type=>:text, :method_name=>"storage_volumes", :value=>""}],
                                                            @action_urls=["http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start",
                                                                          "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                          "http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4"],
                                                            @actions=[["start", "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start"],
                                                                      ["destroy", "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4"],
                                                                      ["create_image", "http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4"]],
                                                            @state="STOPPED">
 
  > i.actions
  => [["start",        "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start"],
      ["destroy",      "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4"],
      ["create_image", "http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4"]]
 
  > i.state
  => "STOPPED"
 
  > i.start!
  > i.state
  => "PENDING"
 
  # and after a little while
  > i.state
  => "RUNNING"

Now you can use the same API to connect to and control openstack:

  $ irb
  > require 'deltacloud'
 
  > c = DeltaCloud.new('admin+admin', 'cloudpass', 'http://localhost:3005/api')
  => <DeltaCloud::API:0x0000000149fc90 @password="cloudpass", @username="admin+admin",
                                        @api_uri=#<URI::HTTP:0x000000014a7530 URL:http://localhost:3005/api>,
                                        @entry_points={:instances=>"http://localhost:3005/api/instances",
                                                       :buckets=>"http://localhost:3005/api/buckets",
                                                       :images=>"http://localhost:3005/api/images",
                                                       :drivers=>"http://localhost:3005/api/drivers",
                                                       :instance_states=>"http://localhost:3005/api/instance_states",
                                                       :realms=>"http://localhost:3005/api/realms",
                                                       :hardware_profiles=>"http://localhost:3005/api/hardware_profiles"},
                                        @features={:instances=>[:user_name, :user_files, :authentication_password]},
                                        @verbose=false, @driver_name="openstack", @api_version="1.0.0">
 
  > c.instances.collect { |i| [i.id, i.name] }
  => [["f6915123-c3de-4a3c-bb0e-e88143ea838f", "2012-09-25 11:32:43 -0400"]]
 
  > c.images.collect { |i| [i.id, i.name] }
  => [["a4180f79-c4bf-4d61-a801-b5990b590fb0", "f16-jeos"]]
 
  > i = c.create_instance('a4180f79-c4bf-4d61-a801-b5990b590fb0')
  => <DeltaCloud::API::Stateful::Instance:0x000000014fe1a0 @base_name="instance", @client=#<DeltaCloud::API:0x0000000149fc90 ...>,
                                                            @url="http://localhost:3005/api/instances/c32688e4-eeb0-468f-a08f-cdf047d36b45",
                                                            @id="c32688e4-eeb0-468f-a08f-cdf047d36b45",
                                                            @objects=[{:method_name=>"stopped?", :type=>:state, :state=>"STOPPED"},
                                                                      {:method_name=>"running?", :type=>:state, :state=>"RUNNING"},
                                                                      {:method_name=>"pending?", :type=>:state, :state=>"PENDING"},
                                                                      {:method_name=>"shutting_down?", :type=>:state, :state=>"SHUTTING_DOWN"},
                                                                      {:type=>:text, :method_name=>"name", :value=>"2012-09-25 14:39:31 -0400"},
                                                                      {:type=>:text, :method_name=>"owner_id", :value=>"admin"},
                                                                      {:type=>:link, :method_name=>"image", :id=>"a4180f79-c4bf-4d61-a801-b5990b590fb0"},
                                                                      {:type=>:text, :method_name=>"image_id", :value=>"a4180f79-c4bf-4d61-a801-b5990b590fb0"},
                                                                      {:type=>:link, :method_name=>"realm", :id=>"default"},
                                                                      {:type=>:text, :method_name=>"realm_id", :value=>"default"},
                                                                      {:type=>:text, :method_name=>"state", :value=>"PENDING"},
                                                                      {:type=>:link, :method_name=>"hardware_profile", :id=>"1"},
                                                                      {:type=>:text, :method_name=>"hardware_profile_id", :value=>"1"},
                                                                      {:type=>:collection, :method_name=>"public_addresses", :values=>[]},
                                                                      {:type=>:collection, :method_name=>"private_addresses", :values=>[]},
                                                                      {:type=>:text, :method_name=>"storage_volumes", :value=>""},
                                                                      {:type=>:collection, :method_name=>"authentication", :values=>{:username=>"removed"}}],
                                                            @action_urls=[],
                                                            @actions=[],
                                                            @state="PENDING">
  irb(main):006:0> i.state
  => "PENDING"
 
  > c.instances.collect { |i| [i.id, i.name] }
  => [["c32688e4-eeb0-468f-a08f-cdf047d36b45", "2012-09-25 14:39:31 -0400"],
      ["f6915123-c3de-4a3c-bb0e-e88143ea838f", "2012-09-25 11:32:43 -0400"]]
 
  # openstack instances autostart
  > c.instances.first.state
  => "RUNNING"

The same API can be used to control ec2, vsphere, gogrid and many others.

imagefactory

To install imagefactory, the core component used to build and push cloud images, simply:

  # yum install imagefactory

You may want to checkout / build the latest imagefactory from source to get all the latest updates:

  # yum install git
  $ git clone git://github.com/aeolusproject/imagefactory.git
  $ cd imagefactory
  $ make rpms
  $ cd imagefactory-plugins
  $ make rpms
  $ rpm -ivh <imagefactory.rpm> <imagefactory-plugins.rpm>

The latest imagefactory also leverages the oVirt SDK to build images:

  $ rpmbuild --rebuild http://repos.fedorapeople.org/repos/aeolus/imagefactory/testing/latest_bindings/ovirt-engine-sdk-3.1.0.3-1.fc14.src.rpm
  $ rpm -ivh <ovirt-engine-sdk.rpm>

To push images to oVirt, mount the NFS export domain you had previous setup.

  $ mount ovirt:/ext/ovirt31export /mnt/ovirt

Two configuration files are read in by imagefactory, one describing the provider which to build images for and the other the credentials to use to push an image to it. For example the rhev provider json & credentials xml files would look like:

  $ cat rhevm.json 
      {"nfs-dir": "/mnt/ovirt", "nfs-path": "/ext/ovirt31export",
       "nfs-host": "ovirt", "cluster": "_any_",
       "api-url": "http://ovirt/api", "timeout": 1800 }
 
  $ cat rhevm.xml 
      <provider_credentials><rhevm_credentials><username>admin@internal</username><password>cloudpass</password></rhevm_credentials></provider_credentials>

And the openstack config files:

  $ cat openstack.json 
    {"glance-host": "openstack", "glance-port": 9292 }
 
  $ cat openstack.xml 
    <provider_credentials><openstack_credentials><username>admin</username><tenant>admin</tenant><password>cloudpass</password><strategy>keystone</strategy><auth_url>http://openstack:5000/v2.0</auth_url></openstack_credentials></provider_credentials>

Setup a template containing the description of the image you would like to build. You may download many useful ones from the Aeolus Templates Repo. There are ones to setup httpd, mysqld and more.

Import the template into imagefactory like so:

  $ sudo imagefactory base_image template.xml

Build the image for a particular cloud provider like so (target image is from last step):

  $ sudo imagefactory --debug target_image --id b55b42f1-aa2d-418f-b635-252e5f343209 rhevm

Push the image to the specified provider like so (image id from last step):

  $ sudo imagefactory --debug provider_image --id "2492c8c2-ada4-4325-a783-da3c37871ba7" --target rhevm rhevm.json rhevm.xml

other components

Aeolus consists of many components which can be used individually or together for a seamless cross-cloud experience. Imagefactory uses one of the most popular components Oz behind the scenes, to install a bare-bones operating system with Just-Enough-OS (JEOS) to bootstrap further installation mechanisms using the tooling native to the local environment. This allows us to generate minimalistic cloud images that are as portable as possible. The Audrey config server is used to orchestrate cross-cloud services and configurations and I have previously posted about snap which uses native tooling to take portable snapshots of cloud instances. Ontop of everything the Conductor RESTful API and Web application is used to manage all this, and configure various components to work with each other with users and an extensive permission schema. Infront of this is a the aeolus-cli which provides a rich command line cloud management interface.

Stay tuned for my next post in the series which will be on the Conductor Web UI and aeolus-cli.

read more

September 24, 2012

Aeolus at the Ohio LinuxFest - Part I Cloud Providers

aeolusThis weekend I head out to Columbus, OH for the Ohio LinuxFest where I will be presenting Aeolus. This is one of the largest community run Linux conferences in the states, and should be my biggest presentation audience yet, so I'm really looking forward to going. The presentation leans heavily on a live demonstration of the software, with a minimal introduction to the topic and architecture at the beginning. This week a few of us in the Aeolus community are going to be blogging / tweeting / etc to try and drive a bit of buzz around our recent work which includes support for more cloud providers (prominently openstack!), various command line utilities and migration tooling, and our community efforts.

The demo consists of two machines, my desktop workstation acting as an 'external' cloud provider, and my laptop running a vm w/ the Aeolus suite on it. Network resources are always unreliable at conferences, so I'm not planning on deploying to ec2 or similar, but rather I have setup OpenStack and oVirt (RHEV) on my desktop, each bonded to their own management network interface, through which they will be controlled via my laptop. The exact same commands and setup I will be demoing works with any cloud provider supported by the Aeolus suite (which is the most comprehensive IaaS cloud management suite to date).

The demo will be run on freshly installed Fedora 17 systems with components from the Fedora package repositories. In retrospect the hardest part of setting everything up was setting up the cloud providers themselves, after which Aeolus was a cinch. It is no surprise through, a Cloud-based computational environment is a very complicated thing, and involves orchestrating many disparate components, each of which may be complicated in its own right. This reinforces the need for a technology such as Aeolus, as even with open frameworks such as oVirt and OpenStack, the end user is often adopting one particular way of doing things, which entails certain benefits and drawbacks. By adopting open standards 100% centred around portability, one is freeing themselves from being tied to a specific implementation and may choose at any point to adopt alternate solutions with minimal effort.

Attached below is my guide on setting up oVirt and OpenStack on a fresh F17 box (after the jump). I plan on posting an Aeolus overview later to demo interaction with the providers setup below.

read more

September 19, 2012

The Art of Aikido

Recently our Sensai sent out an email to the members of our dojo requesting submissions for articles to include in the Fall Newsletter. I've been meaning to write a bit about the sport I love and figured this would be the perfect opportunity and venue to do so.

To step back a bit, Aikido is not only a sport, but also a way of life. Aikido literally means the unification (ai) of the spirit (ki, energy) with the way or path (do). Based on principles of simple physics and mechanics to redirect and use an attacker's force against them, Aikido is the culmination of millenia of human warfare refined to the point where the a practitioner (called an Aikido-ka) learns the optimal placement of one's body, mind, and spirit so as to diffuse an attack and neutralize the enemy. As part of this, one of the prominent lessons learnt by an Aikido-Ka is self control. One must be aware of and in control on one's body and mind at all times to anticipate and affect the course of actions. Only through this continual awareness and relaxation, one can free the body and mind to the point which actions and reactions can occur instantly and with optimal efficiency.

As in the dojo one must be aware at all times to fully appreciate and confront life. While the traditional Japanese saying goes "Cry in the Dojo, Laugh on the Battlefield", many of us are fortunately to live in an age and part of the world where we do not have to face an enemy in combat. Regardless, we as humans still have to content with a continuous battle against our own selves. Our own psyches play against us on a continual basis, fuelling the emotions and feelings that drive the human experience, both the ups and downs, the good times and times of sorrow and anger. It is often too easy to get wrapped up in all this and feel lost / without control, some people living their whole lives this way.

Fortunately the exact same mental training practised by an Aikido-ka to control one's mind and body can be applied to one's self in any situation. By being constantly aware of who we are and what we are feeling, one can continuously examine one's self and alter his/her mental state to the point they desire at any time. With enough practice this becomes completely natural, a hair trigger reaction, whenever one starts feeling anxious or sad, one can train him/herself to completely relax be aware of their current state of being and not let the trivialities of our lives get in the way of the magnificence of existence. After all the human experience is just that, perceived mental interpretations of limited sensory inputs in a universe that is incomprehensible by our standards. Only by freeing the mind can one grasp the true potential of human existence.

Another central concept to master while studying the Art of Aikido, is that of infinity. If such a concept can even be grasped at all! When an attack is launched or received, whether deflected, countered, or neutralized, the Aikido-ka must let his or her energy flow from their center to infinity. If one tries to refrain and hold back or stop the reaction and the natural movement, even for a split second, the attacker can and will gain the advantage. When practiced correctly, there will be no possibility of a counter or resistance. The technique will proceed along the most efficient mathematical path through which all the opponents energy will be redirected to where they have no support and balance, using gravity, leverage, and the Aikido-ka's core / center of strength to drive the counter.

Likewise outside the dojo one must try to continuously expand one's conciousness beyond our limited human perceptions towards infinity.At any given moment we as humans are experiencing only an infestisimle amount of what there is to experience. Furthermore, many people are so lost in their thoughts that they don't take the time to experience every moment in its fullest, taking in the vast amount of energy and signals which we are experiencing at any given time. Even at this very moment you are absorbing light, subconsciously interpreting the many audio clues from the environment around you, even every-continuously adjusting your position due to gravity and friction in minute ways. All forms of energy interact with each other no matter how big or small, and after all all matter is energy! (just moving very slow, E=MC^2).

This brings us back to the the Ki (energy) in Aikido. By unifying one's energy with one's path, one can harmonize and balance their life and by mental training one can begin to detach themselves from the little/trivial things that consume so many peoples lives. That is not to say to not enjoy life pleasures. But everything should be viewed in the scope of what it really is and one should not get too wrapped up over anything. I very much enjoy the Martial Art of Aikido, and am planning to continue training as long as I can. I encourage anyone who might be interested to find a dojo around them, there are many great schools all over the world! Until next time, onigashimas!

read more

August 30, 2012

JSON-RPC via RJR - One API / Many Transports

A few months back I gave a presentation to the Brno Ruby Users group about the JSON-RPC protocol and my implementation RJR, but didn't go into too much detail here (blog post). Recently I've been pushing many updates and improvements, including a sorely needed docs update, and figure now would be a good time to do just that.

The goal was to develop a rpc mechanism that was as extensible and pluggable as possible with the implementation being transport agnostic, eg the developer would be able to satisfy and invoke rpc requests over a variety of transport mechanisms, such as tcp, http, websockets, amqp, and more. This would provide the most outreach for developers, allowing their methods to be invoked in a wide variety of infrastructures and existing systems. It was also important that the handlers be able to determine which transport a request came in on, so as to be able to alter flow-control if desired. To accomplish this, RJR sets a variety of instances variables in the scope of the invoked method handler, things like @rjr_node_type will contain the transport which the request came in on, and other things like @rjr_callback allows the server to send json-rpc request methods back to the client, so long as the transport mechanism remains intact (eg the tcp or websocket remains open, the amqp queue is still valid, etc).

# example server from the RJR documentation:
# define a rpc method called 'hello' which takes
# one argument and returns it in upper case
RJR::Dispatcher.add_handler("hello") { |arg|
  arg.upcase
}
 
# listen for this method via amqp, websockets, http calls
amqp_node  = RJR::AMQPNode.new  :node_id => 'server', :broker => 'localhost'
ws_node    = RJR::WSNode.new    :node_id => 'server', :host   => 'localhost', :port => 8080
www_node   = RJR::WebNode.new   :node_id => 'server', :host   => 'localhost', :port => 8888
 
# start the server and block
multi_node = RJR::MultiNode.new :nodes => [amqp_node, ws_node, www_node]
multi_node.listen
multi_node.join

Since JSON-RPC is a very simple protocol I also wanted to add a mechanism to allow developers to extend the protocol easily, even if this meant that these customizations would only work against nodes running RJR. To do this RJR allows developers to set arbitrary headers to be written to the json-rpc request, so that method handlers and their invokers may process this additional metadata and do what they will with this. For example, a node being used as a server can take method arguments, authenticate them against any backend, and set a 'session-id' header on all subsequent messages. All subsequent client requests will contain this header which is available to the handlers that can authorize the user. (obviously the end user would want to use a secure transport mechanism incorporating ssl to prevent session-hijacking)

# example clients from the documentation
# invoke the method over amqp
amqp_node = RJR::AMQPNode.new :node_id => 'client', :broker => 'localhost'
puts amqp_node.invoke_request('server-queue', 'hello', 'world')
 
# invoke the method over http using rjr
client = RJR::WebNode.new :node_id => 'client'
puts client.invoke_request('http://localhost:8888', 'hello', 'mo')
 
# Invoking json-rpc requests over http using curl
# $ curl -X POST http://localhost:8888 -d '{"jsonrpc":"2.0","method":"hello","params" ["mo"],"id":"123"}'
# > {"jsonrpc":"2.0","id":"123","result":"Hello mo!"}

As far as next steps, flushing out the UDP transport mechanism and continuing to optimize performance are high on my list. At some point I would love to do a complete rewrite in a lower-level language such as C and simply write wrappers / adapters so that methods implemented in higher level languages can be invoked simultanously. But for the time being, RJR serves my purposes and will continue developing that for now.

read more

August 20, 2012

is-it-fedora-ruby - from an idea to an app


Last January I attended and gave a few presentations at the Fedora Users and Developers Conference in Blacksburg, Virgina. The conference was great, we accomplished alot, and among everything else the Fedora rubys-sig met again to discuss packaging, distro, and other issues. During our discussions and all the work up to that point it became aparent that one of the main areas which the community was in need was on the promotional front, we had already done a slew of work to form a stable Ruby and Gem stack in Fedora with several applications relying on it (bringing some sanity to the mad world of ruby dependencies) but our efforts were going unheard of for a large part, lost in the torrent of other upstream ruby community discussions.

To rectify this, I proposed developing a site similar to isitruby19, the now defunct isitmacruby, and the ruby toolbox. This would provide a simple one-stop shop for ruby developers and users alike, where the current stack in Fedora could be easily viewed and discussed, and cross-referenced with the gem metadata stored at rubygems.org. Unfortunately like so many other ideas, this one was put on the backburner in lieu of more pressing work, but would come up periodically as we discussed various ways to grow the Ruby community.

As luck would have it, Fedora was again selected as one of the sponsored organizations for the google summer of code which I saw as an opportunity to drive this project forward. I wrote up the call for proposals, registered as a mentor, and submitted the document. The project received a few applicants, and I felt that Zuhao was the best fit for it so together we flushed out a plan of attack.

The rest was fairly straightforward. Zuhao did the vast majority of the coding over the summer, I submitted a few patches every once in a while to "rubify" things or to implement a small feature. We formally met via IRC on Fridays, and discussed and designed things via email throughout the week. Around the middle of the summer, we deployed the site on a vm on one of my servers, and as the project continued to evolve, we registered the isitfedoraruby.com domain and pointed it there. The vm just contained a clone of the project's git repo so that the latest updates could be deployed very rapidly and we wouldn't have to worry about packaging for the time being. In the near future we will be bundling everything up into formal releases and pushing the packages into the Fedora and Debian release cycles.

All in all, I'm very pleased with Zuhao's progress and the application has been gaining traction amongst the Fedora / Ruby community. We're looking at and are continuing to expand the metadata that is stored in the application and displaying it in unique ways which can't be viewed elsewhere. Make sure to frequent the site as it continues to evolve!

read more

July 19, 2012

First test day - Fedora Jam

The first test day for feodra jam will be next week, on the 26th. We will be looking for bugs, usability problems, broken packages etc. This is the first proper testing for the spin, so it will be mostly just charting the seas to see what needs to be done and what can be improved. 

Permalink | Leave a comment  »

July 16, 2012

GSoC Update

A few months back I wrote about a few projects which I am mentoring for Fedora as part of the Google Summer of Code '12. Now that the summer is about 1/2 way over, I'm pleased to share the progress on the projects, as well as all the hard work that the students' have put into them.

First off Sammy who is working on an interface to deploy to the cloud via specified deployment criteria has made great headway expanding the central Aeolus API to support hardware profiles and instance management. Sammy has also expanded the command line client and utility to support these new capabilities and will be basing the remainder of the work this summer on using these features to deploy to the cheapest and quickest cloud providers.

Nitesh (who has been graciously volunteering his own time outside the GSoC) has been working on building a graphical (QT based) desktop client for Aeolus, which he is hosting on his github repo. The application is looking great, modules have already been completed for provider and instance management from the desktop, and he is looking to continue expanding functionality for end-to-end cloud instance management.

Finaly Zuhao who has been working hard on the isitfedoraruby project has also made great headway, implementing Rake tasks allowing us to import gems and rpms into the database, as well as various frontend interfaces which to display and cross-reference the gem and rpm metadata and other info. Various other features are in progress including the ability to comment and rate specific packages, a section highlighting various projects based on the Fedora / Ruby stack, and a mechanism to more closely integrate w/ rubygems as gems are pushed. You may find the code in Zuhao's github repo, and I've hosted the website on a temp subdomain for demoing purposes. Towards he end of the summer we'll update the site with the latest developments, reserve the is-it-fedora-ruby (or other) domain, and point that at my server.

Any comments, feedback, patches, or feature requests to any of these projects would be more than appreciated. Be sure to stay tuned for more updates as these projects evolve as additional features are added.

read more

July 05, 2012

Getting closer - GSoC Week #x

Not sure which week of GSoC this is, but it's time for an update.

Fedora-jam finally has it's own kickstart and has been uploaded to git. The submission to the spins team has been pushed to the 11th of July. There are still packages beeing tossed around, so i think it would make sense to wait for the discussion to die down a little before we submit it.

I have now also been accepted as a Fedora packager, and i have submitted two packages, monobristol and zita-alsa-pcmi. My next job is to create a program that can switch between pulseaudio and jack by the press of a button. It will probably be written in python, and use qt for the interface (i would prefer to write in c, but i have only ever done GUI programming with python and qt).

Now we have to start testing, figure out what need to be configured. We also need to start setting dates for test days. I'm hoping to arrange two test days. One in late July, and one at the en of GSoC. I will start writing up dates next weke and start spreading the word. 

Permalink | Leave a comment  »

June 28, 2012

The long drag - GSoC Week #4-5

Not much has been happening lately. Most of my time has gone to packageing and trying to get sponsored. Getting a package through review is much harder then it seemed at first. But it seems like i have two packages that are now ready and hopefully by the end of the week i will get sponsored.

We had to push the submission date to the Spins SIG to the 7th of july. I just haven't had time to work properly on the kickstart file. I need to get the list of primary software ready first, which i should get ready sometime today or at the latest tomorrow so that i can send it on the mailinglist so everyone can check that nothing is missing.  

I need to start picking up some speed. Things have been going slow and i think i might be running a little behind schedual. As long as there is a kickstart file to submit by the end of next week, we should be ok. 

I have contacted the SPins SIG aout what can be done after the submission deadline (17th of July), but i still don't feel like i have a proper answer. I think going on the IRC chanel might help, as email can often lead to misunderstandings

Hopefully i will remeber to blog next week. I'm terrible a blogging, never liked doing it, but i'm trying my best :)

 

 

 

Permalink | Leave a comment  »

June 13, 2012

Hitting the ground running - GSoC Week #3

The time has come to get things op and running.

The first IRC meeting about the audio spin was held on Monday. The minutes from the meeting can be read at http://sheep-punchers.info/ftp/fedora-audio/2012/fedora-audio.2012-06-11-15.04.html

Although the attendence was quite low (we were only 3), it was a very productive meeting. I now know more of what has to be done, and when. The spins team has been contacted and they now know about the audio spin, and they gave the exact dates for certain events, which are put into the calendar for the spin. A good way to keep updated with the progress of the spin would be to subscribe to the calendar, as it is updated regurarly.

The wiki pages for the project are also getting an overhaul. The spin itself needs a wiki page, which will be completed on friday when we finally have decided what the spin should be called. The two wiki pages that are there now, will also be made into one, as they contain a lot of the same (but sometimes conflicting) info

The mission for the time being is to start playing around with configuration files (especailly JACK/ALSA/pulseaudio) so we know what has to be configured and what it is possible to use. I have also started on createing the kickstart file, and it will be uploaded as sonn as it has anything interesting in it (don't know where yet)

A lot to to this week, but i would like to see the kickstart ready for the next meeting, which is on the 25th of june, and I would like to see the kickstart file submitted to the spins team by the 1st of july so no time to rest

If anybody wonders where week 2 of GSoC went, it was spent acing my CS exams, and failing my math exam

Permalink | Leave a comment  »

May 26, 2012

Comunitising the comunity - GSoC - week #1

The first week of Goole Summer of Code is now over. As i'm still studying hard at university to finish my exam (this time we are implementing a filesystem on our OS) i have not done to much. Most of my time has gone to get an overview of what has to be done. I have set up a schedual for meetings and for important dates that needs to be kept in mind. If anyone wants to lok at the schedual for the fedora audio spin, you can subscribe to the calendar here

At the same time i have been busy comunitising the comunity. It's dawned on me that i'm supposed to be a driving force behind the fedora audio spin, so keeping mysself and the counity in sync is important. The main thing that is being disccused now is a name for the spin and what desktop enviroment we should use. I'm partial to just ssticking to gnome 3, maybe xfce. Som people have mention Razor Qt, which I will have to try before i make a verdict on it. I love the interaction with the comunity though, and it's fun to get great responses to messages to the mailing list

I also need to get approved for package comminting. I have written my first .spec file for an rpm. It was hard to get into, but with enought wiki reading and staring at examples it did eventually sink in. I was though some sweet commands that really help if anyone is interested in writing rpm packages:

yumdownloader --source <package>
rpm -ivh <resulting src.rpm>

The first command will downoad a source rpm file, the second will unpack it. Gives plenty of info and also lets you view the .spec file of the package.  

Permalink | Leave a comment  »

When in Rome...

Am wrapping up my European trip and am planning on being back in the States at the end of next week. Earlier on this week, my mother (who came for a visit) and I traveled to Rome, Italy to see the city (hover over images for descriptions, right click and then select 'view image' for the full size)

  • On Sunday 05/20 we flew out of Brno, a direct flight to Rome Fiuminco Airport. We arrived pretty late so ended up only going to the hotel and crashing
  • The next day was the Colosseum, Palatine Hill and the Roman Forum, the Basilica of Saint Maria sopra Minerva, the Pantheon, Piazza Navona, and Trevi Fountain. Needless to say we were quite exhausted from all the walking at the end of the day :-)

  • The following morning was the National Museum of Rome and the rest of the day was spent at Vatican City. I have to say I have never seen a collection of artwork quite as impressive as that at the Vatican Museum. Not only that but all in all it is over 9 miles of artwork (yes 9 miles)! All in all, while unbelievably impressive (especially the Raphael Rooms) it was a bit much, towards the end the grandour had not faded (actually quite the opposite) but I was completely museumed out, and couldn't process any more. We also visited Saint Peters Bascillica and several other grand cathedral's during the course of the day.

  • On our third day, Wednesday we got a great view of the city from the Villa Borgese Gardens, walked down the Spanish Steps and then got another panaramic view from the vista atop of Victor Emmanuel's monument. We were pretty much Romed out by this point so besides visiting the Baths of Diocletian on Thursday before our flight back we took it easy for our remaining time there.

All in all the word I feel that best describes the city is "Epic", so many times when rounding a corner when wandering through the city we were facing something incredibly grand and ancient. To any prospective tourists I have a few bits of advice, first off make sure to pace yourself, there is alot too see (no way you are going to cover everything) and you don't want to get burnt out early on. Also I highly recommend taking tours, both the paid tour guide at the Colloseum / Palentine Hill / Roman Forum and the audio tour at the Vatican Museum was well worth it. And lastly / most importantly make sure to enjoy some italian pizza and pasta when your visiting! :-)

Thats all for now, in less than a week I'll be back in the good ole' US of A. Until next time, ciao!

read more

May 08, 2012

Google Summer of Fedora/Aeolus/Ruby

As some of you may know, Fedora was recently accepted, as a participating organization into the Google Summer of Code. Shortly after this was announced, Buddhika (our Fedora GSoC Admin and Coordinator) circulated a RFC for projects to improve the Fedora distribution as part of this effort. I submitted a few proposals to mentor myself, and am pleased to say there quite a bit of interest by various students, a few of which are now officially working on the various topics.

Specifically Samridh Srinath will be working on improving the Aeolus user experience on Fedora, implementing various aspect of the command line interface and core API features needed to drive it, and Zuhao Wan will be working on a site to highlight and promote the Fedora / Ruby experience, providing a simple one-stop-shop to query for Ruby related packages in Fedora and to assist in the migration process.

Unfortunately, not all of the students who applied were able to be accepted into the GSoC due to the limited number of slots we had available. Both Nitesh Narayan and Furhan Shabir are talented students who submitted strong proposals to work on Aeolus but weren't able to secure the official sponsorship. That being said, both have graciously volunteered to donate some time to assist with the project for which I would like to thank them greatly for.

All in all, I'm looking forward to working with these talented students over the course of the summer and beyond. Stay tuned for more updates as the projects progress and the proposals are implemented.

Until next time, happy hacking!

read more

May 06, 2012

JSON-RPC Presentation - Brno Ruby Users Group - 26.04.2012

On the Thursday before last I gave a presentation on the JSON-RPC protocol and my implementation of it to the Brno Ruby Users Group. You may find the slides here as well some code examples in Ruby and Javascript here.

RJR is a simple implementation of the JSON-RPC 2.0 standard, using eventmachine on the backend to serve JSON-RPC requests over a multitude of transport types including amqp, http, websockets, tcp/udp, and more. It permits method handlers to be registered and then invoked via any transport the end-user wishes (and the server admin permits access to).

The context of the request is made available to the handler when it is invoked, so that the developer can manage flow control accordingly and store any additional metadata in the requests and responses to extend the JSON-RPC protocol in any manner they desire.

The source code is freely available on github, any patches/issues/etc would be more than welcome. Happy hacking!

read more

April 30, 2012

Brno Trip Update

How I've been busy in the past few weeks. Between working, sightseeing, and plenty of other activities, I've had no chance till now to sit down and write some content for the blog. Now that I'm approaching the 1/2 way point of my trip, I have to say I've had a phenominal time so far, my European collegues at Red Hat are a group of extremely talented guys who are also very fun to work and hang out with. I'm enjoying my stay in the Czech Republic, the people are friendly, there is plenty to do, and overall a pleasant experience. So far my trip over the last few weeks looked like (hover over the images for descriptions, to view full size right click and select 'view image'):

  • April 3rd/4th: Flight from Syracuse to Newark, NJ to Copenhagen, Denmark to Prague, CZ; Caught the train from Prague to Brno (overall about ~24hrs worth of traveling)
  • Met my coworker Tomas at the Brno train station (he's been a momumental help with the entire trip, a very friendly person and extremely talented engineer). Moved into my apartment here

  • Thu/Fri April 5th/6th: Started working out of the Brno office, met all the other Aeolus and Ruby developers working out of here (all around the same age as me, another +), found my way around town including using the public transports and grocery shopping


  • My first weekend I explored Brno on foot, I saw the historic old town, the Špilberk castle, the Cathedral of St. Peter and Paul, and got a general feel for the area. Brno is a great city, a good size, not too large that it feels crowded, but not too small that there is nothing todo. It is a university town though, so lots of students, but has plenty of great places to hang out, grab a bite or drink, or do various activities
  • The following week primarily consisted of work (it's refreshing to be back in an office working w/ my collegues in person), and attending Aikido class with my collegues Tomas and Petr, both who regularily go. The Sensai here is a city cop that focuses on practical self defense which is different than the more theoretical appoach Sensai Mehter teaches back in Syracuse. Both are good to practice, though I still prefer the dojo back home.
  • This week, we also celebrated Red Hat's Billion Dollar Bash in the Brno office. The facilities here arrainged a very fun scavenger hunt around the city, which culminated at the Brno Museum of Technology right by the RH office. The party there was great, lots of great food and treats, and was good to meet developers and other RH employees working on all sorts of projects @ the company.

  • I also visited the Brno Hackerspace, Base48 this week, which a few of my collegues are members of. It was great seeing yet another Hackerspace in person, and we all shared plenty of stories of our experiences so far. It seems like they're running into many of the same issues we had w/ the SIG, and I assume many others have had worldwide but overall are doing great and working on some cool projects.
  • Was thinking about going to Prague or Vienna on the weekend of the 14th, but ended up just hanging out and relaxing w/ my co-workers after the long week. When I book my flight back from Europe, I'm planning on booking a hotel in Prague for a few days to check out that city.
  • The next week was more work, on Wed 4/18 I decided to swing a last minute trip to Amsterdam in the Netherlands. I left via overnight train on the night of the 19th and arrived the morning of Friday 4/20/12. Stayed until Sunday evening when I caught another overnight train back (had reserved a bed in a couchette both ways). The whole experience was remarkable, Amsterdam truly is a magical city; a melting-pot of European cultures, there is a happy and laid back atmosphere in the air, and a "clock-work" feel to the city (think of those old antique clocks w/ lots of moving gears). There is just so much to see and do (I suggest exploring it on foot), between the canals and city parks, to the museums, to the restaurants, coffee-shops, and nightlife, Amsterdam has it all, and overall based on first impressions of the big cities I visited, Amsterdam takes the cake. I stayed in a hostel Friday and Saturday nights which was an interesting experience, the people I ended up staying with were all really friendly and nothing sketchy or bad happend, but I'm not sure if I'd want to do it again (similarily while I got lots of sleep on the train, it was a weird feeling having other passengers come and go from the cabin I was staying in in the middle of the night when I was sleeping). Unfortunately Amsterdam is a touristy city though, things are expensive and there are lots of tourist attractions (at several points I noticed a stark constast between the old and the new worlds, like a modern carnival which was operating in the plaze of one of the cities biggest / oldest churches). Also when you get down to it, it is a big city, which I can't ever see myself permenatly living in (I tend to prefer smaller cities, and areas with lots of nature and large state parks within a very short distance). All in all though, the weekend in Amsterdam was a great time, and I would eagerly jump at the opportuninty to visit again

  • On Monday of last week the team went go-karting which was a blast followed by ribs and beer :-) One of the best things about the Czech republic is how inexpensive everything is (that is to an American visitor), a good stack of ribs won't cost you more that $6 or $7 USD, tips included. Also a pint of good beer will only run you a dollar or two. Life is good in the Czech republic :-)

  • On the following Wednesday, my UK collegue left to fly back home, so saw him off before working. Had a great time hanging out with Martyn over the last few weeks, and it's good to put a face and memories behind an IRC handle. Work in general has been great, have been getting alot done both on the Aeolus and Ruby/Fedora side of things as well as on my side projects. There have been some exciting updates to everything I've been working on, I've pushed new additions to Snap, have been working with a few students on now-accepted proposals for the Google Summer of Code (which I am now a mentor of, more on this in the near future), have been learning a great deal as far as security and hardening a Rails application is concerned, and am putting a presentation together for the Brno Ruby Users Group (on JSON-RPC and my RJR project), and much more. Look for more details on alot of these in the near future
  • Also went rock climbing for the first time since high school on Thursday with some co-workers. Was a fun time and a great workout, a good whole body excercise (especially the upper body and shoulders). I'm looking forward to going again and perhaps trying to find a place back in Syracuse that I frequent and improve my technique
  • Furthmore I went for a long bike ride around the Brno Lake and Reservoir and then Slacklining nearby on Sunday with the same co-workers. All in all we ended up biking around 35 Kilometes, and saw Veveří Castle in the process, and while our initial attempts on the Slackline weren't all that successful, it was all a great workout and plenty of fun. Definitely more activities that I'm looking forward to doing again when I get the chance (quite possibly this week! :-) ).

All in all so far the trip has been a blast, I've met so many interesting people from many different places. From my collegues who come from all over Europe (the Czech Republic, Slovakia, Hungary, Italy, Russia, the UK, and more), to random people at all the various places I've visited / stayed at, people are genuently friendly and helpful and always willing to swap stories with a foreigner. There are places right out of the movies and yet places which seem unique in their own right and unlike anywhere else. I'm looking forward to the rest of my stay. Besides Prague, I'd like to visit at least one other European city and/or take in some of the country side (I was thinking about renting a car and just driving around the a bit).

Stay tuned for more updates coming soon! Until then, Chow!

read more

April 26, 2012

Google summer of code

I have been selcted to participate in the google sumer of code. I wil be working with a fedora project to create an audio spin of fedora, specially catered to people who want to use studio software without having to set up an entire system by them selves. This is something I would like to use myselff, and i\m really looking forward to it. I would like to thank the wonderfull menotrs over at fedora, they are amazing. As soon as i applied to work with them for GSoC i felt at home. Fedora seems to have an incredible comunity.

From now on I'll be posting about the work I do with the fedora audio spin project. My first task i to package some software. Some great resources to get started with that can be found on the fedora wiki.

Permalink | Leave a comment  »

April 07, 2012

Easter geekage

As a norwegian, easter is a big holiday season. Everyone goes to a cabin, preferably in the middle of nowhere, and stays there for a week. I too had to follow this tradition, so we went to a cabin. There i realised there was no radio, and no alarm clock (both would be handy). I did however bring my laptop, which happens to have 3g. So by clever usage, i set up a cron job to go off at a certain time that would play a radio webstream through vlc. It worked like a charm

tl;dr: I used crontab to set up a radio alarmclock when i was miles away from civilization and only had linux + 3g

Permalink | Leave a comment  »

March 27, 2012

RubyConf India, 2012
This was my/the 3rd RubyConf, first time in Pune. Like last year, I couldn't/wouldn't submit a talk. But I did manage to buy a ticket in time, before they got sold out. My friends were over, and very dear to me that they are, we stayed up Friday night playing UNO, until it was 7am. 8am I was up and reached the Hyatt Regency venue by 9.

Day 1

After the Welcome speech, it was Charles @headius Nutter with his keynote on JRuby and other Ruby VMs. It was great to know how the other ruby VMs were coming along, and how IronRuby was dying. Next was a recorded video keynote by Matz. I started missing the first RubyConf India, where we had the opportunity of having Matz on a Skype video conference. Matz said one place Ruby lags behind Python is SciPy and NumPy, and they would focus on having that in Ruby in the next few years.

After a coffee-break I was attending Ruby CLI talk byNikhil Mungel and Shishir Das. I was a bit put off on having to see only slides in a talk which could have made more interesting by throwing in some cool CLI demos. Next I was attending Tejas Dinkar and Jasim A Basheer's talk on Sandboxing Ruby Code, where they discussed their experiences on developing rubymonk.com.

As usual, lunch was awesome. I met lots of people, unlike my previous RubyConfs, where I operated as a lone shark.

Post lunch (and a bit drowsy) I was attending the Everything Ruby talk by Ajay Gore. It wasn't very interesting. Next up during the Clojure talk by Steven Deobald, I was feeling a bit lost. Noticing nothing interesting lined up next, I headed home.

Day 2

Next morning's keynote was by Mikel Lindsaar. It was a motivational speech, which talked about purpose and value exchange in software development and life in general.

Post coffee, I attended Sou Sheong's talk on Sex, Money and Evolution. That was one of the most interesting talks I have ever seen. He took some virtual bots called roids, and simulated a culture of them after adding external parameters like energy (money), reproduction (sex) and natural selection (evolution). The result was a beautiful visualization of life. Most of it was written using Ruby and R, so I would have been a happy puppy has he demonstrated some of his code. But the source code is on github, so I could have a look at it later. The session was very interactive.

I tried attending the next talk titled "What lies beneath the beautiful code", but the sheer uninterestingness, and the fact that people started walking out ensured I followed suit.

Lunch, again, was full of #win.

Post lunch I attended "Smells and patterns in test/spec code", by Sidu Ponnappa and Aninda Kundu. It talked about the anti-patterns while writing tests. After that I spent some time in the Hack Room discussing some list archive crawler ideas with Anurag, and clicking photos, which I won't publish. No.
Later I found myself in the method_missing should be recursive talk by Matthew Kirk.

Post a couple_of_cutting_chai_and_sandwiches, I was attending the lighting talks. Highlights were nursery_rhymes.rb by Shakthi Kannan and Users == Bugs by Charles Nutter. And that was how RubyConf India 2012 ended.

Some observations

The quality of talks this time wasn't really upto the mark. Some talks were really great, but some were just downright bad. Most of the talks were unrelated to the Ruby language. And many failed to excite the audience. I know it's easy to criticize when I myself didn't submit a talk, but then I will make sure that doesn't happen in RubyConf India 2013.

March 19, 2012

A translation editor for DTD resources
Hi,

I am Shreyank and I will be mentoring the project "A translation editor for DTD resources" under Ankur India.
Since Ankur India has been accepted as a GSoC Organisation, I am hoping to get a lot of queries on what the project is really about. If you are reading this post, it is possible that you have asked me a query, the answer to which I have here.
<background></background>

<background>

Although Ankur India deals mostly with issues relating to the Bengali language, this project idea is not specific to Bengali.

A very common translation workflow consists of translating po files, which in turn are used within applications using gettext. One can use tools like lokalize, gtranslator etc which reads the po files, provides a GUI to help translate the strings and save them back to language specific po files. Thus a file relating to Bengali strings would go into a file called bn_IN.po.

<background\>

Although a lot of translation use cases are covered by po files, some aren't. For example, Mozilla uses DTD and properties files for the purpose. The idea is to extend the existing localisation tool(s) to cover these type of data files.

A logical first step would be to write an engine which is able to read dtd/properties files, take user input for translations and write a new language specific file. This program should be modular and API driven so that it can be quickly used to extend with any existing localisation tool, desktop or web based. On the other hand one should be able to quickly extend the engine to work with other similar data files.

Once that is ready it should be integrated with an existing translation tool, preferably a desktop app so that we have a proof of concept and a finished product which can be used by translators.

The application should be platform independent and can be written in the programmers choice of language, keeping in mind the scalability and portability.

With the above points clear, I'm open to any other suggestions on how to best approach the problem. In case you have any further doubts mail me at shreyankg AT gmail DOT com and I'll be happy to take up questions.

March 18, 2012

Working from Brno, CZ (04/12 - 06/12)

Starting next month I will be spending some time working onsite w/ my teammates at Redhat Brno in the Czech Republic. This will be my first trip to Europe and I am fully looking forward to going, not only have the previous times I spent working from the office (Raleigh, Boston, NYC) been great networking experiences, but this will be the furthest of my more recent adventures yet (I did visit Egypt on a few occasions during my youth) and I am looking forward to seeing the new culture first hand. During this trip I will be renting an apartment with fellow teammate Martyn from the UK who will also be visiting during the month of April.

I am also planning on visiting a few other cities during my stay, including Prague, Vienna, and Amsterdam. If anyone wants me to bring them back any souvenirs, just ask! And make sure to stay tuned for more updates!

read more

March 03, 2012

GTK Programming w/ Python

mmorsi-presentation

This Wednesday, I will be giving a talk at the March Syracuse Linux Users Group meeting on Programming GTK interfaces w/ PyGTK (along w/ Glade and GOBject Introspection). You can find my presentation here, which I made with showoff (source here, more on this in a subsequent blogpost) and is probably my most ahem interesting presentation to this date (at least I like to think so ;-) ).

As far as the $topic matter itself, cross-platform GUI development has never been easier due to a few simple tools. GTK is available on many platforms, and using Glade you can easily whip up an interface that suites your needs. Wiring up event handlers and the backend code is a cinch due to the gobject bindings which gtk provides that we are able to leverage from a Python app.

glade

Along w/ the presentation I'm planning on giving a quick demo on how to perform these steps, creating a simple Desktop based twitter client. You can download the glade xml file here and the python app itself (also see below). Simply run it with $ python syrlug_demo.py but note you will need to have the python twitter library installed (# sudo yum install python-twitter on Fedora).

demo interface

Of course this is just the beginning, you can easily expand the interface by editing the XML in Glade and adding new event handlers to the Python script to do whatever you would like.

import os
import time
import threading
import twitter
from gi.repository import Gtk, Gdk, GObject
 
INPUT_TEXT='twitter username'
 
GLADE_FILE = os.path.join(os.path.dirname( __file__ ), 'layout.glade')
 
# A worker thread to get twitter data
class WorkerThread(threading.Thread):
    terminate = False
    twitter_username = None
    twitter_text = ''
 
    def set_buffer(self, buffer):
      self.buffer = buffer
 
    def run(self):
       while(not self.terminate):
         api = twitter.Api()
         self.twitter_text = ''
         if self.twitter_username != None:
           try:
             for status in api.GetUserTimeline(self.twitter_username):
               self.twitter_text += status.text + "\n"
           except:
             pass
           Gdk.threads_enter()
           self.buffer.set_text(self.twitter_text)
           Gdk.threads_leave()
         time.sleep(5)
 
# The Main Window
class SyrlugDemoWindow:
    def __init__(self):
       self.builder = Gtk.Builder() 
       self.builder.add_objects_from_file(GLADE_FILE, ["syrlug_demo", "twitter_username", "twitter_text"])
       self.window = self.builder.get_object("syrlug_demo") 
       self.username_input = self.builder.get_object("twitter_username") 
 
       self.username_input.set_text(INPUT_TEXT)
 
       self.worker = WorkerThread()
       self.worker.set_buffer(self.builder.get_object("twitter_text").get_buffer());
       self.worker.start()
 
       self.builder.connect_signals(self)
       self.window.connect("destroy", self.close_window)
       self.window.show()
 
    def close_window(self, widget):
        self.worker.terminate = True
        Gtk.main_quit()
 
    def subscribe_button_clicked(self, widget):
        input_text = self.username_input.get_text() 
        if input_text != INPUT_TEXT and input_text != "":
            self.worker.twitter_username = input_text
 
    def twitter_username_focus_in(self, widget, param):
        if widget.get_text() == INPUT_TEXT:
            widget.set_text("")
            #widget.set_visibility(False)
 
    def twitter_username_focus_out(self, widget, param):
        if widget.get_text() == "":
            widget.set_text(INPUT_TEXT)
            #widget.set_visibility(True)
 
if __name__ == "__main__":
    # initialize window
    win = SyrlugDemoWindow()
 
    # start gtk loop
    Gdk.threads_init()
    GObject.threads_init()
    Gdk.threads_enter()
    Gtk.main()
    Gdk.threads_leave()

Now lets see some of those GTK apps!

read more

February 29, 2012

Updated Puppet Web Resource Module

Some of you may recall the Puppet Web Resource type and provider which I wrote for Aeolus a while back. The module has since undergone alot of improvment from various contributers, and was finally accepted and uploaded to the Puppet module forge. You may download it from there and simply drop it into your puppet module load path to access the resource in all its glory!

Some additional features since I last shared this include

  • support for http basic auth,
  • cookies support,
  • extended request invokation conditions (eg 'only invoke request if <condition> is true' and 'invoke request unless <prerequisite web request> is successful, etc),
  • extended response verifications ('contains', 'does not contain', xpath and response status verification),
  • extended logging,
  • and several various other features.

For reference here is the updated web resource type and provider:

lib/puppet/provider/type/web_request.rb

require 'uri'
 
# A puppet resource type used to access resources on the World Wide Web
Puppet::Type.newtype(:web_request) do
    @doc = "Issue a request to a resource on the world wide web"
 
    private
 
    # Validates uris passed in
    def self.validate_uri(url)
      begin
        uri = URI.parse(url)
        raise ArgumentError, "Specified uri #{url} is not valid" if ![URI::HTTP, URI::HTTPS].include?(uri.class)
      rescue URI::InvalidURIError
        raise ArgumentError, "Specified uri #{url} is not valid"
      end
    end
 
    # Validates http statuses passed in
    def self.validate_http_status(status)
      status = [status] unless status.is_a?(Array)
      status.each { |stat|
        stat = stat.to_s
        unless ['100', '101', '102', '122',
                '200', '201', '202', '203', '204', '205', '206', '207', '226',
                '300', '301', '302', '303', '304', '305', '306', '307',
                '400', '401', '402', '403', '404', '405', '406', '407', '408', '409',
                '410', '411', '412', '413', '414', '415', '416', '417', '418',
                '422', '423', '424', '425', '426', '444', '449', '450', '499',
                '500', '501', '502', '503', '504', '505', '506', '507', '508', ' 509', '510'
                ].include?(stat)
          raise ArgumentError, "Invalid http status code #{stat} specified"
        end
      }
    end
 
    # Convert singular params into arrays of strings
    def self.munge_array_params(value)
      value = [value] unless value.is_a?(Array)
      value = value.collect { |val| val.to_s }
      value
    end
 
    newparam :name
 
    newproperty(:get) do
      desc "Issue get request to the specified uri"
      validate do |value| Puppet::Type::Web_request.validate_uri(value) end
    end
 
    newproperty(:post) do
      desc "Issue post request to the specified uri"
      validate do |value| Puppet::Type::Web_request.validate_uri(value) end
    end
 
    newproperty(:delete) do
      desc "Issue delete request to the specified uri"
      validate do |value| Puppet::Type::Web_request.validate_uri(value) end
    end
 
    newproperty(:put) do
      desc "Issue put request to the specified uri"
      validate do |value| Puppet::Type::Web_request.validate_uri(value) end
    end
 
    newparam(:parameters) do
      desc "Hash of parameters to include in the web request"
    end
 
    newparam(:file_parameters) do
      desc "Hash of file parameters to include in the web request"
    end
 
    newparam(:follow) do
      desc "Boolean indicating if redirects should be followed"
      newvalues(:true, :false)
    end
 
    newparam(:store_cookies_at) do
      desc "String indicating where session cookies should be stored"
    end
 
    newparam(:use_cookies_at) do
      desc "String indicating where session cookies should be read from"
    end
 
    newparam(:remove_cookies) do
      desc "Boolean indicating if cookies should be removed after using them"
      newvalues(:true, :false)
    end
 
    newparam(:returns) do
      desc "Expected http return codes of the request"
      defaultto ["200"]
      validate do |value| Puppet::Type::Web_request.validate_http_status(value) end
      munge    do |value| Puppet::Type::Web_request.munge_array_params(value)   end
    end
 
    newparam(:does_not_return) do
      desc "Unexecpected http return codes of the request"
      validate do |value| Puppet::Type::Web_request.validate_http_status(value) end
      munge    do |value| Puppet::Type::Web_request.munge_array_params(value)   end
    end
 
    newparam(:contains) do
      desc "XPath to verify as part of the result"
      munge    do |value| Puppet::Type::Web_request.munge_array_params(value)   end
    end
 
    newparam(:does_not_contain) do
      desc "XPath to verify as not being part of the result"
      munge    do |value| Puppet::Type::Web_request.munge_array_params(value)   end
    end
 
    newparam(:log_to) do
      desc "Log requests/responses to the specified file or directory"
    end
 
    newparam(:only_log_errors) do
      desc "Boolean indicating if we should only log responses which did not pass validations"
      newvalues(:true, :false)
    end
 
    newparam(:if) do
      desc "Invoke request only if the specified request returns true"
    end
 
    newparam(:unless) do
      desc "Invoke request unless the specified request returns true"
    end
 
    newparam(:username) do
      desc "HTTP authentication username"
    end
 
    newparam(:password) do
      desc "HTTP authentication password"
    end
 
end

lib/puppet/provider/web_request/curl.rb

require 'fileutils'
 
# Provides an interface to curl using the curb gem for puppet
require 'curb'
 
# uses nokogiri to verify responses w/ xpath
require 'nokogiri'
 
class Curl::Easy
 
  # Format request parameters for the specified request method
  def self.format_params(method, params, file_params)
    if([:get, :delete].include?(method))
      return params.collect { |k,v| "#{k}=#{v}" }.join("&") unless params.nil?
      return ""
    end
    # post, put:
    cparams = []
    params.each_pair      { |k,v| cparams << Curl::PostField.content(k,v) } unless params.nil?
    file_params.each_pair { |k,v| cparams << Curl::PostField.file(k,v)    } unless file_params.nil?
    return cparams
  end
 
  # Format a url for the specified request method, base uri, and parameters
  def self.format_url(method, uri, params)
    if([:get, :delete].include?(method))
      url = uri
      url +=  ";" + format_params(method, params)
      return url
    end
    # post, put:
    return uri
  end
 
  # Invoke a new curl request and return result
  def self.web_request(method, uri, params = {})
    raise Puppet::Error, "Must specify http method (#{method}) and uri (#{uri})" if method.nil? || uri.nil?
 
    curl = self.new
 
    if params.has_key?(:cookie) && !params[:cookie].nil?
      curl.enable_cookies = true
      curl.cookiefile = params[:cookie]
      curl.cookiejar  = params[:cookie]
    end
 
    if params.has_key?(:username) && !params[:username].nil?
      curl.username = params[:username]
    end
 
    if params.has_key?(:password) && !params[:password].nil?
      curl.password = params[:password]
    end
 
    curl.follow_location = (params.has_key?(:follow) && params[:follow])
    request_params = params[:parameters]
    file_params    = params[:file_parameters]
 
    case(method)
    when 'get'
      curl.url = format_url(method, uri, request_params)
      curl.http_get
      return curl
 
    when 'post'
      curl.url = format_url(method, uri, request_params)
      curl.multipart_form_post = true if !file_params.nil? && file_params.size > 0
      curl.http_post(*format_params(method, request_params, file_params))
      return curl
 
    when 'put'
      curl.url = format_url(method, uri, request_params)
      curl.multipart_form_post = true if !file_params.nil? && file_params.size > 0
      curl.http_put(*format_params(method, request_params, file_params))
      return curl
 
    when 'delete'
      curl.url = format_url(method, uri, request_params)
      curl.http_delete
      return curl
    end
  end
 
  def valid_status_code?(valid_values=[])
    valid_values.include?(response_code.to_s)
  end
 
  def valid_xpath?(xpath="/")
    !Nokogiri::HTML(body_str.to_s).xpath(xpath.to_s).empty?
  end
 
end
 
# Puppet provider definition
Puppet::Type.type(:web_request).provide :curl do
  desc "Use curl to access web resources"
 
  def get
    @uri
  end
 
  def post
    @uri
  end
 
  def delete
    @uri
  end
 
  def put
    @uri
  end
 
  def get=(uri)
    @uri = uri
    process_params('get', @resource, uri)
  end
 
  def post=(uri)
    @uri = uri
    process_params('post', @resource, uri)
  end
 
  def delete=(uri)
    @uri = uri
    process_params('delete', @resource, uri)
  end
 
  def put=(uri)
    @uri = uri
    process_params('put', @resource, uri)
  end
 
  private
 
  # Helper to process/parse web parameters
  def process_params(request_method, params, uri)
    begin
      error = nil
      cookies = nil
      if params[:store_cookies_at]
        if File.exist?(params[:store_cookies_at])
          File.truncate(params[:store_cookies_at], 0)
        else
          FileUtils.touch(params[:store_cookies_at])
        end
        cookies = params[:store_cookies_at]
      elsif params[:use_cookies_at]
        cookies = params[:use_cookies_at]
      end
 
      # verify that we should actually run the request
      return if skip_request?(params, cookies)
 
      # Actually run the request and verify the result
      result = Curl::Easy::web_request(request_method, uri,
                                       :parameters => params[:parameters],
                                       :file_parameters => params[:file_parameters],
                                       :cookie => cookies,
                                       :follow => params[:follow],
                                       :username => params[:username],
                                       :password => params[:password])
 
      result_body = result.body_str.to_s
 
      verify_result(result,
                    :returns          => params[:returns],
                    :does_not_return  => params[:does_not_return],
                    :contains         => params[:contains],
                    :does_not_contain => params[:does_not_contain] )
 
      result.close
 
    rescue Exception => e
      error = e
      raise Puppet::Error, "An exception was raised when invoking web request: #{e}"
 
    ensure
      unless result.nil?
        log_response(:result => result_body,
                     :method => request_method, :uri => uri,
                     :puppet_params => params,  :error => error)
      end
      FileUtils.rm_f(cookies) if params[:remove_cookies]
    end
  end
 
  # Helper to determine if we should skip the request
  def skip_request?(params, cookie = nil)
    [:if, :unless].each { |c|
      condition = params[c]
      unless condition.nil?
        method = (condition.keys & ['get', 'post', 'delete', 'put']).first
        result = Curl::Easy::web_request(method, condition[method],
                                         :parameters => condition['parameters'],
                                         :file_parameters => condition['file_parameters'],
                                         :cookie => cookie, :follow => params[:follow])
        result_succeeded = true
        begin
          verify_result(result, condition)
        rescue Puppet::Error
          result_succeeded = false
        end
        return true if (c == :if && !result_succeeded) || (c == :unless && result_succeeded)
      end
    }
    return false
  end
 
  # Helper to verify the response
  def verify_result(result, verify = {})
    verify[:returns]          = verify['returns']          if verify[:returns].nil?          && !verify['returns'].nil?
    verify[:does_not_return]  = verify['does_not_return']  if verify[:does_not_return].nil?  && !verify['does_not_return'].nil?
    verify[:contains]         = verify['contains']         if verify[:contains].nil?         && !verify['contains'].nil?
    verify[:does_not_contain] = verify['does_not_contain'] if verify[:does_not_contain].nil? && !verify['does_not_contain'].nil?
 
    if !verify[:returns].nil? &&
       !result.valid_status_code?(verify[:returns])
         raise Puppet::Error, "Invalid HTTP Return Code: #{result.response_code},
                               was expecting one of #{verify[:returns].join(", ")}"
    end
 
    if !verify[:does_not_return].nil? &&
       result.valid_status_code?(verify[:does_not_return])
         raise Puppet::Error, "Invalid HTTP Return Code: #{result.response_code},
                               was not expecting one of #{verify[:does_not_return].join(", ")}"
    end
 
    if !verify[:contains].nil? &&
       !result.valid_xpath?(verify[:contains])
         raise Puppet::Error, "Expecting #{verify[:contains]} in the result"
    end
 
    if !verify[:does_not_contain].nil? &&
       result.valid_xpath?(verify[:does_not_contain])
         raise Puppet::Error, "Not expecting #{verify[:does_not_contain]} in the result"
    end
  end
 
  def log_response(params)
    method  = params[:method]
    uri     = params[:uri]
    result  = params[:result]
    error   = params[:error]
    puppet_params = params[:puppet_params]
 
    if puppet_params[:log_to]
      return if puppet_params[:only_log_errors] == :true && error.nil?
      logfile = puppet_params[:log_to].strip
      exists = File.exists?(logfile)
      isfile = File.file?(logfile) || (!exists && (logfile[-1].chr != '/'))
      if !isfile
        FileUtils.mkdir_p(logfile) if !exists
	      logfile += puppet_params[:name]
      end
 
      f = File.open(logfile, 'a')
      f.write("=====BEGIN=====\n")
      f.write(Time.now.strftime("%Y-%m-%d %H:%M:%S"))
      f.write(" #{method} request to #{uri}\n")
      f.write(result.to_s)
      f.write("\n=====END=====\n\n")
      f.close
    end
  end
end

As always, Happy Puppeteering!

read more

January 16, 2012

Fudcon Blacksburg Report

Just got back from Fudcon Blacksburg. As always, it was a great time. Arrived after driving down (10 hrs) Thursday evening and crashed quickly after checking into my accommodations (The Inn at Virginia Tech). Friday, the first day, was hackfests, I attended the Fedora Infrastructure meeting and held a few sessions around Aeolus where I talked with passer by's about the project, my Snap subproject and what people were looking for out of an open source IaaS cloud abstraction platform. Specifically at FUDCon I aimed to find out how people wanted to interface w/ the cloud on the Fedora desktop and how they wanted to use the Aeolus stack. We ended up modifying Snap a little to add some command line options and began
messing around w/ mycloud a c level cloud computing API I started throwing together to meet people's needs. Dinner that night was at a good indian place.

The second day was the barcamp sessions, I volunteered to do another session on Aeolus, which went well, there were plenty of newcomers interested in the projects and I was able to share what was in development and upcoming things on the platform. After lunch several of us from the ruby-sig met to hold the now traditional ruby-sig meeting at FUDCon (we held one in Toronto, one was held in Milan, and now this was the third at Blacksburg). We got alot done including:

  • tested out the new Ruby 1.9.3 repository (messed around w/ getting Aeolus working on MRI 1.9.3)
  • discussed updates to the new packaging guidelines, and updates to the the ruby stack
  • looked into possibly using macros to assist w/ building gems for multiple ruby interpreters on Fedora (jeroen volunteered to start investigating thing one, this would be a feature coming furthur down the pipeline)
  • discussed the rvm / bundler ecosystems, and again how to drive and market the Fedora philosophy of a single supported stack while delivering the ultimate development platform for the ruby-community
  • looked into continuing building tools and applications which to enhance the Ruby experience on Fedora, I volunteered to build a isitfedoraruby website (akin to itisruby19.org) to demo and highlight fedora on ruby and to start reaching out to the cloud-sig and other Fedora communities to promote and highlight the product/platform.
  • continuing to extend / expand / improve gem2rpm, polisher, and other gem/rpm interoperability tools

Next I attended the session on OpenStack, a popular build-your-own-cloud platform (one of the platforms supported by deltacloud) and learned a bit more about the internals and how it is constructed. Finally the lightning talks session was the last on Saturday, and a must for any FUDCon attendee. There were many pitches, including several for new fedora components including hosted instances of mock infra apps, a discussion on what motivates and how to drive open source contributions, a demonstration on how to inject code into running Python processes using "parasite" by lmacken, an overview of foss@rit by remmy dekaukusburg and the narcissus web visualizations by ralph bean. I ended up pitching mycloud and snap again as well as my own Nethack Encyclopedia project, inviting fellow explorers of the mazes of menace to bring the guide :-)

FUDPub was fun, as always a good time, featuring bowling and lots of tastey food and held at the Blacksburg student center this time around.

I'm writing this from my hotel room Sunday morning as I pack up and prepare for the 10hr drive hack. Overall a great time, have plenty to keep me busy going forward, and am looking forward to the next FUDCon!

read more

January 03, 2012

Forthcoming: MRI Ruby 1.9 in Fedora

The Fedora Ruby SIG is in the process of updating the main Ruby package in Fedora to Ruby 1.9. We hope to get it in by Fedora 17, while shipping a Ruby 1.8 compatability package for those who wish to continue using the older version (though more and more upstream projects are phasing this out). Alternatively, since JRuby is now in Fedora, end-users may opt to utilize that interpreter for both Ruby 1.8 and 1.9 support.

The Ruby 1.9 effort is being led by Vit Ondruch who has done alot of the packaging work and submitted the initial guidelines draft, which we are currently discussing/revising on the mailing lists. They are largely an extension of the existing guidelines to better nail things down and cover various issues that have come up over the years, but all are welcome to join the discussion and share their comments / opinions on the best way to move forward.

A Ruby 1.9.3 testing repository has been setup for those who wish to try the new packages out. Feel free to share any feedback or issues on list.

read more

January 01, 2012

Book Review: Altered Carbon

Having finished my long trek through Anathem a little while back, I was left craving for another epic sci-fi tale. Living up to his previous classic, Snow Crash, Neal Stephenson's Anathem is an amazing thrill-ride, full of intense action, profound philosophy (drawing on thousands of years of mathematics, science, and metaphysics), and overall an unbelievable story. Stephenson is truly a remarkable author, and while he deals with very-deep topics, there is a good chance that anyone will be able to find at least one of his worlds that draws them in. Surprisingly I couldn't get into Cryptonomicon when I tried not too long ago, but admittedly I didn't get terribly far into it before I gave up and might have to give it another go. There are plenty of reviews of Anathem online, so I will leave it up to my readers to read more about it through those, but needless to say I highly recommend the novel.

I ordered two books with relatively good reviews off of Amazon, Hyperion (a space opera) and the second being Altered Carbon by Richard K Morgan, a nitty gritty cyberpunk tale similar to Snow Crash. I tried the former first, but again try as I might I could not get into it, so I decided to give Altered Carbon a go, and was very pleasantly surprised.

Altered Carbon is the tale of Takeshi Kovacs as he investigates the 'murder' of Laurens Bancroft, a very rich and powerful business magnate in futuristic San Francisco, now called Bay City. By this point in time, actual real deaths are fairly uncommon, as it is a widespread practice to store one's identity, memories, and conscience 'on stack', which is able to be downloaded into temporary bodies known as 'sleeves' at any point in time after. Morgan's vivid and rich imagery paints a world that lives up to any cyberpunk setting portrayed to date, including perhaps the ultimate cyberpunk novel, Neuromancer by William Gibson. Chalk full of concepts such as virtual-realities, technology based augmentations, drug induced psychological and physiological alterations, interstellar travel, conspiring artificial intelligences, oriental philosophy, intense martial arts action, and much more, Kovacs' investigation keeps the reader glued to the edge of his/her seat and makes it hard to put the book down even for a second.

The novel is a good read, not so challenging that it will be lost on anyone (unlike Anathem and Snow Crash which are complete mind-benders), but also not too simplistic that it will bore. The chapters are fairly short and to the point, making for a good airplane or subway read, while the overall story fits together and flows nicely. Without spoiling any more, I highly suggest it to anyone that is into the genre, it won't disappoint!

read more

December 23, 2011

Messing around w/ HTML5: drag-n-drop

In my last html5 blog post we explored using the canvas to recreate a simple version of pong. This time around we'll be using the drag-and-drop features of HTML5 for a simple example. Note I'm pulling in the JQuery UI library to take care of alot of the details and provide a easy-to-use interface to use drag-and-drop.

In this example, we have a list of images which we'd like to pull into a common area. The thing is, the original list should be preserved, once dragged-in, an image should not disappear from its original location.

This is accomplished by using jquery to set the images in the list to being draggable (this registers the necessary event handlers), and to make the common area droppable, eg able to receive draggable objects. JQuery allows us to set the 'helper' attribute, which defines the element being dragged around, to the predefined 'clone' value which makes a copy of the original element. From there we expand the container div to accept only the elements we want to drag in, and override the 'drop' method to set various properties allowing us to position and style the dropped element any way we would like.

All in all the effect is the following:

<script type="text/javascript"> $(function(){ $('.jquery_drag_drop_example_img').draggable({ helper: 'clone', }); $('#jquery_drag_drop_example_container').droppable({ accept: '.jquery_drag_drop_example_img', drop: function(event, ui){ var newitem = $(ui.draggable).clone(); $(this).append(newitem); $(newitem).addClass("jquery_drag_drop_example_img_clone"); $(newitem).removeClass("ui-draggable jquery_drag_drop_example_img"); $(newitem).draggable({containment: 'parent'}); $(newitem).css('position', 'absolute'); $(newitem).css('top', ui.position.top ); $(newitem).css('left', ui.position.left); //alert(event.dataTransfer.getData('text')); } }); }); </script>

And the source producing this:

<div id="jquery_drag_drop_example_container"
     style="width: 300px; height: 300px; background: #C0C0C0; float:left;">
</div>
<div style="background: #FFFF99; float:left; width: 20%; margin-left: 10px;">
<ul style="list-style-type: none; list-style-position: inside;">
  <li><img class="jquery_drag_drop_example_img" src="example_0.png" /></li>
  <li><img class="jquery_drag_drop_example_img" src="example_1.png" /></li>
  <li><img class="jquery_drag_drop_example_img" src="example_2.png" /></li>
  <li><img class="jquery_drag_drop_example_img" src="example_3.png" /></li>
  <li><img class="jquery_drag_drop_example_img" src="example_4.png" /></li>
</ul>
</div>
<div style="clear: both;"></div>

<script type="text/javascript">
$(function(){
  $('.jquery_drag_drop_example_img').draggable({
    helper: 'clone',
  }); 
  $('#jquery_drag_drop_example_container').droppable({
    accept: '.jquery_drag_drop_example_img',
    drop: function(event, ui){
      var newitem = $(ui.draggable).clone();
      $(this).append(newitem);
      $(newitem).addClass("jquery_drag_drop_example_img_clone");
      $(newitem).removeClass("ui-draggable jquery_drag_drop_example_img");
      $(newitem).draggable({containment: 'parent'});
      $(newitem).css('position', 'absolute');
      $(newitem).css('top', ui.position.top ); 
      $(newitem).css('left', ui.position.left);
      //alert(event.dataTransfer.getData('text'));
    }   
  }); 
});
</script>

Thats it until next time. Hope everyone enjoys the holidays!

read more

December 20, 2011

Snap! Updates

Alot of great development on Snap! recently. Here are some updates (see the commit log for the full list):

- incorporated full test harness
- many documentation improvements / fixes
- many code cleanups / optimizations
- starting adding ability to migrate snapshots between OS's (for example Ubuntu to Fedora and vice-versa)
- pushed snap into Fedora! (simply install via 'yum install snap' on Fedora15+ and RHEL6, should be pushed into Debian / Ubuntu soon)

Stay tuned for more updates!

read more

December 19, 2011

Various Quick Tips

Just a bunch of random quick tips, posted together for conciseness

---------------------------

Passwordless SSH'ing as root

When writing the test harness for Snap I needed to setup a few vms which I could ssh into as root automatically, without being prompted for a password. True, I could've setup a ssh key and install that on the vms, but instead for simplicity I wanted to be able to ssh in w/out requiring any credentials (yes insecure, but these VMs are isolated and the test harness makes sure they are shutdown post-invocation).

On Fedora/Ubuntu this is accomplished by:
- editing /etc/ssh/sshd_config an setting 'PermitEmptyPasswords yes'
- Ubuntu requires one additional config change, to /etc/pam.d/sshd, namely comment out '@include common-auth' and replace with 'auth required pam_unix.so nullok'
- restarting ssh: 'sudo service sshd restart'
- setting sshd to come up automatically 'sudo chkconfig --levels 35 sshd on'
- finally delete the root user's password 'sudo passwd -d root'

Now you can ssh root@<ip_address> to login as root w/out a password (obviously use with caution).

Incidentally this is also a means to get a root shell on Ubuntu, which by default does not allow this, requiring the user to run all privileged commands through 'sudo'

---------------------------

xbacklight

I've run into a problem w/ the backlight not resuming normal operation after my laptop has been dock'd. This seems to be a common issue, even on the Linux friendly Lenovo Thinkpad laptops.

After some debugging I was able to resolve this by installing the 'xbacklight' package and running 'xbacklight -set 100'.

---------------------------

C Data Structure Visibility Practices

I've been getting back into C programming (more on this soon). Of course w/ a lower-level language (C is actually a high level language, though usually considered low-level, but I digress) there are many approaches / ways to do things, especially concerning data visibility. Studying the libvirt library header file, I found one useful standard is to:

- define public data structures, eg structures which can be manually instantiated by the user, as you normally would, eg

typedef struct _myData{
  ...fields...
} myData;
 
typedef myDataPtr *myData;

- define private data structures, or data which can only be instantiated by calling one of your methods, w/ forward declerations in the public header, with the private definitions residing in an internal-only header/implementation file, eg

typedef struct _privateData privateData;
typedef privateDataPtr *privateData;
 
privateDataPtr createPrivateData(...params...);

- define public data structures refering to data managed / manipulated by an external entity, as 'info' structures, (these will often be associated w/ private data in some way) eg

typedef struct _externalDataInfo {
  ...fields about data managed elsewhere...
} externalDataInfo;
 
typedef externalDataInfoPtr *externalDataInfo;
 
externalDataInfoPtr getExternalDataInfo(privateDataPtr private_data);

---------------------------

Per Host SSH Key

Many thanks to jkeating for this pro-tip. To set which ssh key to use on a per-host basis, simply create a ~/.ssh/config file (chmod'd to 600) containing the following contents:

Host <hostname>
IdentityFile <path-to-ssh-key>

Also optionally specify 'User <username>' to not have to prefix your remote username to the hostname when logging in w/ a username different than your local one.

---------------------------

Thats all folks!

read more

December 10, 2011

Plasma PackageKit integration approved as a Fedora 17 feature

I am proud to announce that my Plasma PackageKit Integration GSoC 2011 project has been approved as an official Fedora 17 feature.

In other news, I have recently laid the first stone for a port of my work to KDE Frameworks 5, by getting support for a plasma5 namespace into PackageKit.


December 05, 2011

libvirtd default network on a vm
I've run into this issue so many times that I figure I'd share the solution incase anyone else has a similar problem. If you are running libvirtd on a vm which was in itself started by libvirt, make sure you change the network address of the default libvirt network on the vm. Or alternatively simply run the following command to shut it down:

$ sudo virsh net-destroy default

This is because otherwise the addresses of the libvirt created networks running on the vm and your host machine will clash and cause conflicts / routing errors.

Hope this helps!

read more

October 18, 2011

Planning FUDCon Pune



I'm in charge of Wireless and Charging Points at FUDCon, Pune.

http://openetherpad.org/fudcon-pune-planning contains the ongoing planning details.

September 28, 2011

GSoC is over, but my code lives

The GSoC deadline is now over. I passed the final evaluation and got my payment, still waiting for the t-shirt and certificate though. But the fact that the deadline has passed doesn’t mean I stopped maintaining the code. This week, the telepathy-kde-presence-applet has landed in Rawhide, which made me find and fix a few issues in my GSoC work:

  1. The RPM auto-Provides script got confused by plasma-scriptengine-applet-declarative.desktop in kde-runtime, which has a X-KDE-PluginInfo-Name different from the X-Plasma-API (the name is qmlscript, the API is declarativeappletscript). The dependency extractor was preferring the X-KDE-PluginInfo-Name, but that name is not used for script engines, only the API is relevant. This is fixed by this kde-settings commit, in kde-settings-4.7-8.fc17.
  2. The data engine dependency extraction did not support declarativeappletscript QML code. I updated the patch to add support for that feature.
  3. The plasma-dataengine-depextractor was passing a relative path to KDesktopFile, which misinterpreted it as relative to applnk rather than to the current directory. I fixed the patch to use QDir::absoluteFilePath instead of QDir::filePath.
  4. The plasma-dataengine-depextractor was requiring the user or script calling it to explicitly pass the scripting API/language used through an --api argument. That information is already provided by the .desktop file, which is required anyway, so I dropped the useless parameter and changed the code to detect the scripting API automatically. (If anybody is already using the plasma-dataengine-depextractor: please just remove any -a language or --api language argument you’re passing, it is now all detected automatically.)

Unfortunately, I haven’t been able to build the updated data engine dependency extraction patch into Rawhide yet because the kdelibs build fails due to an unrelated issue with isnan.

UPDATE: glibc has been fixed in Rawhide, and kdelibs-4.7.1-4.fc17 was built successfully.


September 20, 2011

Pycon India 2011, Pune
So I attended Pycon India for the first time in three years it has existed. It was held in Bangalore the previous two years, and I was being lazy.
Ratnadeep was put up at my place. I had originally decided not to attend the tutorials, but then since Ratnadeep wanted to attend, I decided to show up as well.

Day 1

Tutorials on Friday was a bit boring, considering they were long, and I wasn't exactly the student types. Navin Kabra had a session about Web API Programming using Python where I lingered for a while. I did however attend a part of Anand Chitipothu's session on Functional Programming. I liked the slides he made using Landslide. Must check that out sometime.

Day 2

Raymond Hettinger had his keynote on why Python is awesome. His talk highlighted what makes Python different from other languages. Then Anand Chitipothu had a talk on pyjs, which is a way to compile python code into JS. The talk was good, especially the AST parts. After lunch I went to attend Sajjad's talk on developing Android apps using Python. Track 2 was jampacked with more people waiting outside, so the talk was moved to Track 1 which was more spacious. Noufal then took the stage with his talk on using Emacs as a Python IDE, which then got side tracked to org mode. The rest of the day I spent chit-chatting.

Day 3

Sunday meant, my talk had to be delivered soon, and my slides were somewhat ready. Nevertheless I spend some (a_lot_of) time retouching the slides and doing up code examples. I was there, half-attentive to Jace's talk on the LastUser Service. It looked interesting and I am planning to use it for the FUDCon COD installation. After lunch and coffee, (and an indecent where I mixed up my laptop with Anurag's, and spend multiple attempts trying to log-in into his system), I gave my talk, on developing Django Apps based on REST Architecture (slides| video). For what it's worth, it wasn't very well received by the audience. Also this was the largest audience I ever had. Met people and photographed then the rest of the day.

All in all a good conference. Kudos to the organising team.

Below are some of the pictures I took at the conference.

<object height="375" width="500"> <param name="flashvars" value="offsite=true&amp;lang=en-us&amp;page_show_url=%2Fphotos%2Fshreyankg%2Fsets%2F72157627711726378%2Fshow%2F&amp;page_show_back_url=%2Fphotos%2Fshreyankg%2Fsets%2F72157627711726378%2F&amp;set_id=72157627711726378&amp;jump_to="/> <param name="movie" value="http://www.flickr.com/apps/slideshow/show.swf?v=107931"/> <param name="allowFullScreen" value="true"/> <embed allowfullscreen="true" flashvars="offsite=true&amp;lang=en-us&amp;page_show_url=%2Fphotos%2Fshreyankg%2Fsets%2F72157627711726378%2Fshow%2F&amp;page_show_back_url=%2Fphotos%2Fshreyankg%2Fsets%2F72157627711726378%2F&amp;set_id=72157627711726378&amp;jump_to=" height="375" src="http://www.flickr.com/apps/slideshow/show.swf?v=107931" type="application/x-shockwave-flash" width="500"></embed></object>

August 20, 2011

Plasma/RPM/PackageKit GSoC work now in Rawhide

My GSoC 2011 work has now fully entered Rawhide, i.e. what will eventually become Fedora 17 and later.

The PackageKit portions of the work have already been in Rawhide for a while:

I have just imported the remaining portions of the work into Rawhide:

  • kde-settings-4.7-4.fc17 includes the RPM dependency generators.
  • I have rebuilt kdelibs, kdebase-runtime, kdebase-workspace and kdeplasma-addons to pick up the new Provides. Other packages will automatically pick up Provides and script engine Requires the next time they are rebuilt. (For data engine Requires, you will want to run the new plasma-dataengine-depextractor in kdelibs-devel during the build process, unless/until the upstream metadata.desktop includes an X-Plasma-RequiredDataEngines entry.)
  • kdelibs-4.7.0-3.fc17 includes all the libplasma portions of my GSoC project (as backported patches).

The new features should now be fully testable in Rawhide.

Update: I built kde-settings-4.7-5.fc17, which fixes showstopper bug #732271. (I also fixed a related issue in libplasma in kdelibs-4.7.0-4.fc17, but that fix should not be strictly required because the only thing I use Plasma::PackageMetadata::serviceType() for is to figure out the precise type of script engine to verify the presence of, and if I don’t have a service type, I just conservatively look for any script engine for the language, which should be fine as they tend to be packaged together anyway.)


August 17, 2011

Plasma + PackageKit: The last piece of the puzzle

I have just submitted a patch which implement automatic scanning of the source code of Plasma-related packages (e.g. widgets) for required data engines.

For packages in scripting languages and distributed through Open Collaboration Services (OCS), this is fully automatic and triggered from Package::installPackage. If an X-Plasma-RequiredDataEngines entry is present in the .desktop file (even if empty), the dependency extraction is not run and the explicitly provided information is trusted instead.

For native distribution packages, my patch adds a tool called plasma-dataengine-depextractor which can be run at any time during the build process and which adds the dependency information (the X-Plasma-RequiredDataEngines entry) to the relevant .desktop file.

Authors of plasmoids are encouraged to run plasma-dataengine-depextractor and/or fill in X-Plasma-RequiredDataEngines manually. (Please note that the list is expected to be comma-separated.)

Of course, the automatic scanning is not perfect; in particular, it will not detect convoluted ways to load data engines (e.g. if the name is a variable), and it may have false positives in some corner cases (commented-out use of a data engine, some other function called dataEngine and taking a string literal). However, I expect it to work well in practice, and if it doesn’t work, there’s always the possibility to explicitly add the X-Plasma-RequiredDataEngines entry.

This is the final portion of my GSoC 2011 project, so the project can now be considered 100% complete.


August 10, 2011

It works: Plasma now looks up missing components through PackageKit!

Sorry, I haven’t blogged for a while, but rest assured that I’m still alive and coding. :-) I passed the midterm evaluation, and the final evaluation is approaching. So what do we have in the store?

  • My patches to PackageKit, Apper and gnome-packagekit got pushed upstream. The changes to the PackageKit core even trickled down all the way to Fedora 15 (and of course Fedora 16 and Rawhide). I have backported the Apper changes to the Rawhide kpackagekit package.
  • I added support for Plasma to automatically prompt for the installation of missing script and data engines the first time something attempts to use them. This patch got pushed upstream.
  • I submitted another patch for review, which makes Plasma request the installation of required script engines (always, as this is covered by existing metadata) and data engines (if specified in the metadata through a new X-Plasma-RequiredDataEngines entry) as soon as a package (usually a widget) is installed through Open Collaboration Services (OCS), rather than only when the user attempts to add the widget to some containment (desktop, panel etc.). This patch is pending upstream feedback.
  • I also wrote an RPM auto-Requires script based on the same metadata as the above patch. This ensures script and data engine dependencies will be automatically detected in future Fedora packages.

Now the only remaining work item on my project plan is automatic scanning for required data engines. (Script engines are already completely handled by the above patches.)


June 26, 2011

Small API change to the PackageKit support for Plasma service lookup

Richard Hughes requested a small change to the PackageKit session API used to look up Plasma services before accepting the added function upstream: instead of a dedicated InstallPlasmaResources method, the org.freedesktop.PackageKit.Modify D-Bus interface got a generic InstallResources method, which allows installing all the existing resource types as well as Plasma services, and any other resource types added in the future. It takes a type parameter, which must be set to "plasma-service" in our case.

With those tweaks, the PackageKit change and the gnome-packagekit change are now committed to upstream git. For Apper (KPackageKit), which is our main target, there is a patch for Apper master on Review Board and a backport for KPackageKit 0.6.x on the packagekit mailing list. Hopefully, Daniel Nicoletti will have some time to review the patch soon. If not, he suggested on IRC that I can just commit the patch as is and he’ll refactor it later if needed, so that’s my fallback plan.


June 20, 2011

PackageKit support for Plasma service lookup

I just submitted patches to the packagekit mailing list which implement a new InstallPlasmaResources method in the org.freedesktop.PackageKit.Modify D-Bus interface. There are patches for PackageKit itself, as well as for Apper (both master and KPackageKit 0.6.x) and gnome-packagekit. See my post to the packagekit mailing list for more details.

This completes the next step in my GSoC 2011 project. Now I can start working on using the new interface in Plasma.


June 05, 2011

Automatic Plasma RPM Provides

This week, I worked on my first task in my GSoC project: automatically generating RPM Provides for Plasma services from Plasma’s metadata (in .desktop file format). This turned out to be much easier than I had initially expected, also because my mentor Rex Dieter already had a working proof of concept I was able to start from. I adapted the code to use RPM 4.9′s new plug&play dependency generation functionality and, after discussing this with my mentor, changed the format of the Provides to plasma4(servicetype-name).

My work on Provides and (later on) Requires can be found in the gsoc-2011-plasma-rpm branch of kde-settings on Fedora Hosted. It can be installed separately, but it might be merged into upstream RPM or Plasma once finished.

Today, I tested the script on the kdeplasma-addons package. The resulting Provides look like this:

[kevin@laptop64 x86_64]$ rpm -qp --provides kdeplasma-addons-4.6.3-1.fc15.x86_64.rpm
kcm_krunner_audioplayercontrol.so()(64bit)  
kcm_krunner_charrunner.so()(64bit)  
kcm_krunner_spellcheck.so()(64bit)  
kcm_plasma_runner_events.so()(64bit)  
krunner_audioplayercontrol.so()(64bit)  
krunner_browserhistory.so()(64bit)  
krunner_charrunner.so()(64bit)  
krunner_contacts.so()(64bit)  
krunner_converter.so()(64bit)  
krunner_katesessions.so()(64bit)  
krunner_konquerorsessions.so()(64bit)  
krunner_konsolesessions.so()(64bit)  
krunner_kopete.so()(64bit)  
krunner_mediawiki.so()(64bit)  
krunner_spellcheckrunner.so()(64bit)  
plasma-applet_systemloadviewer.so()(64bit)  
plasma-dataengine-comic = 4.6.3-1.fc15
plasma-dataengine-microblog = 4.6.3-1.fc15
plasma-dataengine-ocs = 4.6.3-1.fc15
plasma-dataengine-potd = 4.6.3-1.fc15
plasma-runner-events = 0.3.0-100
plasma4(applet-comic) = 1.0
plasma4(applet-dict) = 1.0
plasma4(applet-fileWatcher) = 1.0
plasma4(applet-frame) = 1.0
plasma4(applet-incomingmsg) = 1.0
plasma4(applet-kdeobservatory) = 0.2
plasma4(applet-knowledgebase) = 0.1
plasma4(applet-konqprofiles) = 1.0
plasma4(applet-konsoleprofiles) = 1.0
plasma4(applet-lancelot_launcher) = 1.9.5
plasma4(applet-lancelot_part) = 1.5
plasma4(applet-leavenote) = 1.0
plasma4(applet-life) = 0.8
plasma4(applet-magnifique) = 0.1
plasma4(applet-mediaplayer) = 0.1
plasma4(applet-nowplaying) = 1.0
plasma4(applet-opendesktop) = 0.1
plasma4(applet-opendesktop_activities) = 0.1
plasma4(applet-pastebin) = 1.0
plasma4(applet-plasma_applet_notes) = 1.0
plasma4(applet-plasma_applet_rtm) = 0.1
plasma4(applet-previewer) = 1.0
plasma4(applet-qalculate) = 0.7.2
plasma4(containment-org.kde.griddesktop) = 0.1
plasma4(containment-org.kde.groupingdesktop) = 0.1
plasma4(containment-org.kde.groupingpanel) = 0.1
plasma4(dataengine-comic)  
plasma4(dataengine-kdeobservatory) = 0.1
plasma4(dataengine-microblog)  
plasma4(dataengine-potd)  
plasma4(dataengine-rtm) = 0.1
plasma4(packagestructure-comic) = pre0.1
plasma4(runner-CharacterRunner) = 1
plasma4(runner-audioplayercontrol) = 0.1
plasma4(runner-browserhistory) = 1.0
plasma4(runner-katesessions) = 1.0
plasma4(runner-konquerorsessions) = 1.0
plasma4(runner-konsolesessions) = 1.0
plasma4(runner-kopete_runner) = 0.1
plasma4(runner-krunner_spellcheck) = 1.0
plasma4(runner-org.kde.datetime) = 1.0
plasma4(runner-org.kde.events_runner) = 0.3.0
plasma4(runner-techbase) = 1.0
plasma4(runner-unitconverter) = 1.0
plasma4(runner-wikipedia) = 1.0
plasma4(runner-wikitravel) = 1.0
plasma_applet_bball.so()(64bit)  
plasma_applet_binaryclock.so()(64bit)  
plasma_applet_blackboard.so()(64bit)  
plasma_applet_bookmarks.so()(64bit)  
plasma_applet_bubblemon.so()(64bit)  
plasma_applet_calculator.so()(64bit)  
plasma_applet_charselect.so()(64bit)  
plasma_applet_comic.so()(64bit)  
plasma_applet_dict.so()(64bit)  
plasma_applet_eyes.so()(64bit)  
plasma_applet_fifteenPuzzle.so()(64bit)  
plasma_applet_fileWatcher.so()(64bit)  
plasma_applet_frame.so()(64bit)  
plasma_applet_fuzzy_clock.so()(64bit)  
plasma_applet_incomingmsg.so()(64bit)  
plasma_applet_kdeobservatory.so()(64bit)  
plasma_applet_knowledgebase.so()(64bit)  
plasma_applet_kolourpicker.so()(64bit)  
plasma_applet_konqprofiles.so()(64bit)  
plasma_applet_konsoleprofiles.so()(64bit)  
plasma_applet_lancelot_launcher.so()(64bit)  
plasma_applet_lancelot_part.so()(64bit)  
plasma_applet_leavenote.so()(64bit)  
plasma_applet_life.so()(64bit)  
plasma_applet_luna.so()(64bit)  
plasma_applet_magnifique.so()(64bit)  
plasma_applet_mediaplayer.so()(64bit)  
plasma_applet_microblog.so()(64bit)  
plasma_applet_news.so()(64bit)  
plasma_applet_notes.so()(64bit)  
plasma_applet_nowplaying.so()(64bit)  
plasma_applet_opendesktop.so()(64bit)  
plasma_applet_opendesktop_activities.so()(64bit)  
plasma_applet_paste.so()(64bit)  
plasma_applet_pastebin.so()(64bit)  
plasma_applet_plasmaboard.so()(64bit)  
plasma_applet_previewer.so()(64bit)  
plasma_applet_qalculate.so()(64bit)  
plasma_applet_rssnow.so()(64bit)  
plasma_applet_rtm.so()(64bit)  
plasma_applet_showdashboard.so()(64bit)  
plasma_applet_showdesktop.so()(64bit)  
plasma_applet_spellcheck.so()(64bit)  
plasma_applet_timer.so()(64bit)  
plasma_applet_unitconverter.so()(64bit)  
plasma_applet_weather.so()(64bit)  
plasma_applet_weatherstation.so()(64bit)  
plasma_applet_webslice.so()(64bit)  
plasma_comic_krossprovider.so()(64bit)  
plasma_containment_griddesktop.so()(64bit)  
plasma_containment_groupingdesktop.so()(64bit)  
plasma_containment_groupingpanel.so()(64bit)  
plasma_engine_comic.so()(64bit)  
plasma_engine_kdeobservatory.so()(64bit)  
plasma_engine_microblog.so()(64bit)  
plasma_engine_ocs.so()(64bit)  
plasma_engine_potd.so()(64bit)  
plasma_engine_rtm.so()(64bit)  
plasma_packagestructure_comic.so()(64bit)  
plasma_potd_apodprovider.so()(64bit)  
plasma_potd_epodprovider.so()(64bit)  
plasma_potd_flickrprovider.so()(64bit)  
plasma_potd_oseiprovider.so()(64bit)  
plasma_potd_wcpotdprovider.so()(64bit)  
plasma_runner_datetime.so()(64bit)  
plasma_runner_events.so()(64bit)  
plasma_wallpaper_mandelbrot.so()(64bit)  
plasma_wallpaper_pattern.so()(64bit)  
plasma_wallpaper_virus.so()(64bit)  
plasma_wallpaper_weather.so()(64bit)  
kdeplasma-addons = 4.6.3-1.fc15
kdeplasma-addons(x86-64) = 4.6.3-1.fc15

(The Provides starting with “plasma4(” are the new, automatically detected ones.)

The next task is going to be asking upstream Plasma for feedback on this work and on how to proceed, because my next tasks will require closer interaction with Plasma upstream.


April 25, 2011

GSoC 2011 Proposal Accepted

I have been notified today that my proposal was accepted! I’m looking forward to working on KDE Plasma Dependency Generation and PackageKit Integration during the summer. As posted in a previous blog post, my full proposal (minus telephone number) can be found in the Fedora Wiki.


April 08, 2011

Submission deadline passed

So the submission deadline has now passed. I proofread my proposal again today and decided it was fine as is.

Now I have to wait a little more than 2 weeks for the notification of the results (hopefully acceptance).


April 06, 2011

Proposal submitted

I submitted a first version of my proposal. The submitted proposal (without my telephone number ;-) ) can be found at:
https://fedoraproject.org/wiki/User:Kkofler/GSoC_2011_Proposal

The title of my proposal is: KDE Plasma Dependency Generation and PackageKit Integration.


April 04, 2011

Google Summer of Code 2011

I’m applying to participate as a student in the Google Summer of Code 2011, with Fedora as the mentoring organization.

If accepted, this blog will cover the progress of the project.

Expect more information (proposal etc.) to be posted soon.