Fedora People

Musings on the linux audio stack

2012-05-16T19:51:45+00:00

I spent some free time today getting caught up on the large backlog of phonon-gstreamer bugs. Towards the end, I started to have delusions of grandeur: Imagine a phonon-gstreamer codebase that doesn’t require supporting a zillion different audio frameworks, and instead belays that task to something that I don’t have to maintain.

My question here, is how many people would throw a fit if phonon-gstreamer dropped support for ALSA and OSS, and forced everyone to use pulseaudio by way of GStreamer’s excellent pulseaudio support?

Hold on, lower your pitchforks for a minute. Let us consider the audio framework landscape in the modern world:

Pulseaudio is the One True Way for audio playback in Gnome
For 90% of the support questions we handle in #kde-multimedia, the solution is “use pulseaudio”.
Pulseaudio can handle using OSS, ALSA, Bluetooth, or whatever your audio output is, through one consistent entry point
It is a total headache to figure out any bugs in your audio when music goes from Amarok->Phonon->Phonon-GStreamer->(ALSA, OSS, Pulseaudio, god knows what)->Speakers->Earholes

Additionally, I really don’t feel like testing phonon-gstreamer on all those different kernel-level interfaces with exotic setups every time I fix a bug and am afraid I’d introduce another twelve. The PulseAudio folks seem to do a fantastic job at that already. Phonon isn’t meant for real-time playback or production studio quality audio. Thats what Jack is meant for.

I can’t think of a good reason why we shouldn’t stand on the shoulders of giants by making PulseAudio handle all the hard stuff on Unix involving massaging PCM formats, equalizers, matching playback category with output device, enumerating outputs both real and virtual, volume control, etc.

If you can, leave a comment on this post. I’m not making an official statement saying that I’m definitely removing ALSA and OSS support from phonon-gstreamer, I’m merely asking for feedback to see what can be done to fix things at all levels in the audio stack.

<g:plusone href="http://wm161.net/2012/05/16/musings-on-the-linux-audio-stack/" size="tall"></g:plusone>

Zeitgeist improvements with genetic algorithms

2012-05-16T16:43:29+00:00

Continuing with my previous post about the Zeitgeist team’s improvements with regards to speed, there’s a nifty tool in the sources I wrote yesterday that uses a genetic algorithm to find the slowest queries you can throw at the engine.

If you’re not familiar with genetic algorithms, here’s a brief review of how they work:

Start off with an array of numbers, with each index corresponding to a particular attribute of the problem.
Evaluate the fitness of that genome
Simulate evolution of the successful genomes by crossing, mutations, etc, just as you would with real DNA chromosomes

In the case of this Zeitgeist tool, the chromosome refers to a query, and each allele (index of the array) refers to an attribute of the query. Here’s a relevant comment from the sources:

# Chromosome to data mapping:
# 0, 1 - Timerange begin and end. If both are zero, we use timerange.always()
# 2 - The search type. Anything over 30 is a dead individual.
# 3-5 - Specify template properties. Anything besides 0 and 1 is dead.
# 3 - Specify a subject interpretation
# 4 - Specify a subject manifestation
# 5 - Specify an event actor

Using the super cool pyevolve library, implementing a genetic algorithm is super easy:

def eval_func(chromosome):
  query = buildQuery(chromosome)
  if query is None:
    return 0

  start = time.time()
  results = engine.find_events(*query)
  overall = (time.time() - start)
  return (results["find_events"]*2+results["find_event_ids"]*4+results["get_events"])*1000

genome = G1DList.G1DList(6)
genome.evaluator.set(eval_func)
ga = GSimpleGA.GSimpleGA(genome)
ga.evolve(freq_stats = 1)
query = buildQuery(ga.bestIndividual())
assert query is not None
print query, len(engine.find_events(*query))

Let it run for a long while on a big database, and you end up with a query that takes forever. Due to how evolution works, it isn’t the longest running query, but it is certainly one that takes a long time.

<g:plusone href="http://wm161.net/2012/05/16/zeitgeist-improvements-with-genetic-algorithms/" size="tall"></g:plusone>

First evah FUDCon crowd-sourced book

2012-05-16T15:19:53+00:00

Join us as we make history.

3 days. 400 participants. One book.

Participate in the collaborative creation of the first-ever crowd-sourced Fedora book during FUDCon KL, using only your web browser and the power of community*.

Details at the FUDCon Docs Hack website.

There will be swag.

*contribute from anywhere in the world with an internet connection.

Final Donation Drive for the Liberated Pixel Cup!

2012-05-16T14:45:34+00:00

Thanks to Mozilla, the Liberated Pixel Cup got a huge donation, but we’re still about 800 dlls short of the goal, and the deadline’s this Friday. So go ahead, and donate if you haven’t already.

If you’d like a small glimpse of the Sprites that are already being worked on, Lunpa made a small, interactive web demo using naked male and female sprites, as well as some bats, butterflies, houses and other tiles that will be used.

This is only one sample though, there’s going to be many games and many more sprites made for the contest, but I hope that the demo’s enough to get you to chip in for the contest.

lm-sensors deployment, issue with sensors-detect

2012-05-16T14:39:26+00:00

If you wish to deploy lm-sensors on multiple diferent linux machines issue is with sensors-detect what ask you quite a few questions.

I asked on IRC and got few suggestions. First one is if you wish to accept defaults which sensors-detect suggest you need just to press [ENTER] all the time. To automate this use this one liner:

(while :; do echo ""; done ) | sensors-detect

If you wish to override defaults and answer YES to all questions then use this oneliner:

yes "" | sensors-detect

Signed Authentication and Authorization

2012-05-16T14:31:05+00:00

Openstack Keystone currently operates on-line validation for Tokens. Once a token is issued out, each of the systems presented with the token has to check the validity of the token with the Keystone server. This makes Keystone the highest traffic service in an Openstack deployment. Using Cryptographic Message Syntax (CMS) we can generated a token that can be verified using public key cryptography instead of making a network call. Here’s a proof-of-concept example using the command line tools.

For authorization, we need a user name. For authentication, we need a tenancy and a set of roles that the user has in the tenancy.
Authorization cannot be for ever, so we need an expiration date/time. Here is a simple JSON representation of that data.

{"user": "ayoung",
"tenant": "coop-city",
"role": ["hallmonitor, groundskeeper"],
"expiry":"18JUL2012"
}

Note that it has the Roles listed in it. By cryptographically verifying this document, not only does it remove the need to validate the token with Keystone, but it removes the need to go back to keystone to fetch the roles. Thus, authentication and authorization information are linked in one document.

The following assumes you have a signing certificate. See the steps here to generate one if you need it.

To Sign auth_token.json run:

cmsutil -S -d alias -N ayoung -i auth_token.json -o auth_token.p7s

The signed message is put into auth_token.p7s, a file extension that alludes to the PKCS 7 standard.

This includes all of the certificates. Use a tool to view the contents in human readable form:

 /usr/lib64/nss/unsupported-tools/derdump -i auth_token.p7s

The output is big. To simply read the message back, use:

cmsutil  -D -i auth_token.p7s  -d alias/ -h1

This gives back the additional file, along with the validated signer:

SMIME: 	level=1.2; type=signedData; nsigners=1;
		signer0.id="Adam Young"; signer0.status=GoodSignature;
	level=1.1; type=data;
{"user": "ayoung",
"tenant": "coop-city",
"role": ["hallmonitor, groundskeeper"],
"expiry":"18JUL2012"
}

To simply check if the document is valid, drop the h1 argument. If to test it against an NSS database that does not have the CA cert in it, you get:

# cmsutil  -D -i auth_token.p7s  -d altdb

signer 0 status = SigningCertNotTrusted
cmsutil: problem decoding: Peer's Certificate issuer is not recognized.

To make this usable as a replacement for the current token scheme, we could use something like Base64 encoding:

[ayoung@ayoung cmstest]$ base64 auth_token.p7s
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAaCAJIAEbXsidXNl
ciI6ICJheW91bmciLCAgCiJ0ZW5hbnQiOiAiY29vcC1jaXR5IiwgIAoicm9sZSI6IFsiaGFsbG1v
bml0b3IsIGdyb3VuZHNrZWVwZXIiXSwKImV4cGlyeSI6IjE4SlVMMjAxMiIKfQoAAAAAAACgggIG
MIICAjCCAWugAwIBAgICCSkwDQYJKoZIhvcNAQEFBQAwFDESMBAGA1UEAxMJTXkgSXNzdWVyMB4X
DTEyMDUxNTIwNDcxNFoXDTEyMDgxNTIwNDcxNFowUzELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAk1B
MREwDwYDVQQHEwhXZXN0Zm9yZDEPMA0GA1UEChMGUmVkSGF0MRMwEQYDVQQDEwpBZGFtIFlvdW5n
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC7lNCEwdZMHIClZJ9f84R1tfgEjtZV2H+Nx1OR
eqeK+WYvVXSVJ61g/Vl3ponv8L/qRiqbuyP7Nfr7ogfM2OJuDOiWsyfHJSIS29NVOb5f6+oPw39Q
+fN7NSssTEr6UFz4d1mXl85iJXK+K7x5TGFQowA6po102YCj6tjesGPODQIDAQABoyQwIjAgBgNV
HSUBAf8EFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwDQYJKoZIhvcNAQEFBQADgYEAt105m7XYBugD
y+PXYq7R2XaAdyJGCBl4yq3Xz3qmfYmpP8wVTNy5j2dacuopq+W4DB0CeGo5+sYGAiRSgF8vNx6/
wMt3+EExdJO+IJl9QMRB6ooRzSMJAIH3b3jXOWln9TV6AHLHER8NWoNZq3moSYp4fO8tUVgerDKJ
7TdBP/IxggFaMIIBVgIBATAaMBQxEjAQBgNVBAMTCU15IElzc3VlcgICCSkwCQYFKw4DAhoFAKCB
lzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMCMGCSqGSIb3DQEJBDEWBBRBECaXMyVVMoW2yonU
kWwriSqs5TApBgkrBgEEAYI3EAQxHDAaMBQxEjAQBgNVBAMTCU15IElzc3VlcgICCSkwKwYLKoZI
hvcNAQkQAgsxHKAaMBQxEjAQBgNVBAMTCU15IElzc3VlcgICCSkwDQYJKoZIhvcNAQEBBQAEgYAV
UCVEsgfo3SK5p5ZF98uWgoIjmyCksS7mULJMT/ZJZJqIyTVY6AmXWCgxzKOHqrzWCansSilYAYQr
aeEgCYjX9GvSxHFmCQQzYM2buzJr5GjoPZJ0osOulUzWX8Nzq5Y51DAcWtU8c+06Ezbu5q5YjBno
2fTEWwYcemr9m/SA7AAAAAAAAA==

The break is due to how it is displayed in the browser, but that is really one long stream of characters. The value can be sent just like the current tokens, in the -X-Auth-Token header. Now a remote service can validate the token by reversing the Base64 ENCODING AND using cmsutil or the corresponding APIs.

cat auth_token.p7s.base64  | base64 -d |  cmsutil  -D -i auth_token.p7s -d alias/

There are still a few wrinkles to iron out.

The token is a little longer than it needs to be, due to the certificate being embedded. For Keystone, remote systems would be better off fetching the certificate once from the Keystone server, and just matching the serial number in the CMS token.
The current scheme of using the token in the URL would probably not work very well with the long Base64 encoding above, but that system has other known issues, and is likely to be replaced shortly with a scheme that will instead put the token into the body of the document. However, even that step will be unnecessary with certificate based verification.
Signing documents from Python currently requires calling out to additional processes. The Python-NSS maintainer is aware of the issue, and is looking into making a Python-appropriate set of bindings for the CMS calls in Python-NSS.
Since there is no revocation of the tokens, their lifespan should be kept short. 10-30 minutes is probably appropriate. The default value will require some consensus
Adding roles for a user will require getting an additional token. The old token will still be valid,but will not allow the user to do any actions that require the new role.
The code that checks roles does not currently know about the token. It will require a small amount of code changes to make sure that the token is available to this code.
While this scheme is backwards compatible with the current auth scheme, the reverse is not true. Short of checking the length of the token, there is no way to confirm that a token is a cryptographically signed document as opposed to the alpha-numeric cookie in current usage. One approach for transition may be to test reading the document, and, upon failure, fall back to the online protocol.

Generating a Signing Cert using certutil

2012-05-16T13:53:45+00:00

Imagine a locked room with a big window. If I am the only person with a key to room, and I tape a poster up inside the window, everyone can read it, and everyone can state with a pretty high degree of certainty that I was the person that I put up the poster. This is analogy to how you can use PKI to sign a document.

In order for the message signature to be verifiable, we generate a pair of keys, one that is kept private, and one that is made public. Data encrypted with the private key can then be decrypted with only the corresponding public key. So, to sign a message, it is encrypted with the private key.

There are ways to make this more efficient, such as signing a hash of the message, but the principal is the same.

I am going to use the same set of cryptographic tools that Mozilla provides for securing web traffic. These tools fall under then name Network Security Services or NSS for short.

The first step is to provide a Certificate Authority (CA) Certificate. This is a certificate used to sign other certificates. This is actually supposed to be part of a chain. In the browser, there are a list of CA certs that are accepted by default. For our purposes here, I am going to self sign one. Figuring out how to set this up for your organization has been left as an exercise to the reader.

We need a place to put the certificate. For NSS, this is called an NSS database. I am going to create one in a directory I call alias, because that is the name of the NSS database used in Apache HTTPD. To generate:

mkdir alias
certutil -N -d ./alias/

Now create a self-signed CA certificate. Type the command:

certutil -S -s "CN=My Issuer" -n myissuer -x -t "C,C,C" -1 -2 -5 -m 1234 -d alias/

You will be prompted to type. This is used to generate entropy, or randomness, for the underlying cryptography.

Then you will be prompted to select aspects of the certificate. A menu of options like this:

		0 - Digital Signature
		1 - Non-repudiation
		2 - Key encipherment
		3 - Data encipherment
		4 - Key agreement
		5 - Cert signing key
		6 - CRL signing key
		Other to finish

Type each number in order. 0 1 2 3 4 5 6. The menu will reappear after each.

Yes, it is annoying. I didn’t write the tool.

then type any number to finish

So the number 9 works.

 9
Is this a critical extension [y/N]?
y
Is this a CA certificate [y/N]?
y
Enter the path length constraint, enter to skip [<0 for unlimited path]: >
Is this a critical extension [y/N]?
y

Not done yet: Next menu.

		0 - SSL Client
		1 - SSL Server
		2 - S/MIME
		3 - Object Signing
		4 - Reserved for future use
		5 - SSL CA
		6 - S/MIME CA
		7 - Object Signing CA
		Other to finish

For this we want 5 6 7 then again pick 9 to exit.

 > 9
Is this a critical extension [y/N]?
y

The security conscious folks out there are squirming in their seats reading this, because a self-signed CA cert is BAD under most circumstances. However, this is proof-of-concept type stuff. You always need to get a certificate signed by a CA…this is just the simple way to do it for development.

Now that we have a CA cert, we need to generate a Key. With NSS, you can generate the Key and the certificate signing request in one step:

 certutil -R -s "CN=Adam Young, O=RedHat , L=Westford, ST=MA, C=US" -p "617-555-1212" -o mycert.req -d alias

You see the same entropy generation prompt as before.

You can list the keys using:

certutil -K -d ./alias/

Which produces output along the lines of:

certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services"
< 0> rsa      99f6d153a007f8fb2d79312d454749eb92e4db99   NSS Certificate DB:myissuer
< 1> rsa      b8472a65af1c2da900fb0eb953d9c278d615a580   NSS Certificate DB:ayoung

Note that there is one in there for the CA certificate we produced before.

To sign the key run the following:

 certutil -C -m 2345 -i mycert.req -o mycert.crt -c myissuer -d ./alias/ -6

Here is the menu you are presented:

		0 - Server Auth
		1 - Client Auth
		2 - Code Signing
		3 - Email Protection
		4 - Timestamp
		5 - OCSP Responder
		6 - Step-up
		Other to finish

The import options are 1 and 3, as we are going to use the same mechanism as we would use to sign email. The private and public keys are held in the NSS database. OK, we finally have a useful certificate and key pair that we can use to sign a document. To Sign auth_token.json run:

cmsutil -S -d alias -N ayoung -i signme.txt -o signed.p7s

The signed message is put into signed.p7s. The file extension alludes to the PKCS 7 standard that CMS grew out of.

Forcing Google to show a mobile OAuth approval page

2012-05-16T12:08:00+00:00

I did not find this mentioned anywhere on the Internet. Hopefully someone might find this helpful.

This stupid bug had me confused for the last couple of months. It turns out that the Google-specific btmpl parameter does not work as documented. Neither do they recognize the Nokia N9′s user-agent as a mobile phone browser.

Not unsurprisingly, Android phones work, and so does the iPhone.

Here are the relevant user-agents, in case someone out there needs them.

Nokia N9: Mozilla/5.0 (MeeGo; NokiaN9) AppleWebKit/534.13 (KHTML, like Gecko) NokiaBrowser/8.5.0 Mobile Safari/534.13

HTC Wildfire: Mozilla/5.0 (Linux; U; Android 2.2.1; en-us; HTC Wildfire Build/FRG83D) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

iPhone 4S: Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3

It appears as if they are grepping for the names of well-known mobile platforms instead of just looking for “Mobile”. I am surprised because Google does not seem to have an Android branded or iPhone branded page as some people (like Facebook) have.

Fedora Reloaded Podcast: May 11, 2012

2012-05-16T11:38:58+00:00

Updated: Fixed the links to this week's podcast

This week we talk about the upcoming Malaysia FUDCon - the Fedora Users and Developers Conference, and the experimental Docs Hack that we'll be doing there.

Fedora Reloaded May 11, 2012 "I love the freedom" Ogg Vorbis edition
Fedora Reloaded May 11, 2012 "Give me convenience or give me death" mp3 edition

Subscribe to the podcast feed

Making P2 working with RPM - last milestone

2012-05-16T10:30:00+00:00

Maybe you remember my last post about making P2 and RPM working robustly together. Having the Eclipse Platform M7 packaged in Fedora, and Fedora frozen, I have finally found time to look at that work again.

So, first of all, I was surprised how poor programmer was the past me. Then I have fixed a couple of hardcoded paths, got the solution working again, and even managed to prepare and install my first RPM-wrapped feature into the modified version of Eclipse without breaking the ownership of files.

Ownership of files is a pretty important aspect of Fedora security mechanism - a file installed by RPM into /usr should not be changed because otherwise the verification of system will fail - the system will be believed to be compromised. P2 changes many files in Eclipse installation if it only is allowed to do so.

After the installation - Eclipse successfully started with my new RPM feature installed. But then I have discovered a hole in my logic. And then my logic appeared to be like a cheese:

I totally forgot about the fact that once I generate a profile from an assembled installation, P2 will start working as usual, and I will not be able to uninstall anything. I'm not sure if you know it, but P2 is an API to control the bundles.info file, which is

only.

P2 does not care if your bundles.info got changed. It will overwrite it.

My attempts to solve that problem made the scope of changes growing very fast. I even created wiki page and put there some things that need to be taken into account.

I'm sorry folks, especially for those haunted by issues similar to:

You will not get a revamped version of P2. I do not think it is possible and I'm officially tossing this idea. You have to be patient - there is no instant cure for those issues. Eclipse Shared Installation works poorly out of the box (see bug 358471), and things get even worse if you take into account additional restrictions that Fedora puts on P2.

In the meantime - what do you think about preventing Eclipse from being run by root by accident? It is terribly easy to do that on Linux, but then all your dropins stuff goes into master profile and you cannot revert that.

Kermit: new videos documentation

2012-05-16T10:00:47+00:00

With @LouisCoilliot we are preparing some new videos that will be used to explain kermit’s main functionalities; basically will be a doc extension. Naturally you could find all the videos on the KermIT website (we are actually preparing the new version).

KermIT: execute basic operations

KermIT: Post installation steps

KermIT: Server details

Stay tuned for many others KermIT news!!

Fedora 15 kernel plans

2012-05-16T01:23:54+00:00

The F15 release has been an oddity and somewhat of a headache from a kernel perspective. We've stuck with the strange 2.6.4x naming scheme to prevent userspace from breaking on the 3.0 scheme, and for the past 5 months it seems that almost nobody is testing things from updates-testing. It's as if nobody is using F15 at all. That does mean a fairly low number of new bugs reported, but it still has maintenance burden for us and we get concerned when updates sit around for weeks without getting to stable. Maybe we worry too much. Anyway...

With F17 rapidly approaching release, we briefly discussed the plan for the final month of F15. Looking at the usage rate, the upstream kernel status, and the overall timeframe before F15 EOL, we decided that we'll stick with the 3.3.y kernel series for the remainder of its lifetime. There should be one or two more 3.3.y releases upstream as the 3.5 merge window opens and we'll pick up a number of fixes that way. I'll be on F15 duty for the final month and will be backporting any important fixes that aren't picked up by the upstream stable series.

This will break commonality with F16 and F17 in base kernel version. That commonality has served us very well throughout the F15 lifetime, but rebasing it to 3.4 is of limited value at this point. I would be surprised if a 3.4 rebase even got to the F15 stable updates repo before it went EOL to be honest. Fortunately, we can stick with the 3.3.y series without having to worry about it drying up or going unsupported upstream for long durations of time.

So there you have it. F15, in my opinion, was a decent release overall and I used it for quite a while. I believe F16 was better and F17 improves on that yet again, but that should be the norm for releases in Fedora. For those of you still on F15, you might want to look at upgrading soon. If you don't, at least throw some bodhi karma our way so we can put F15 out to pasture gently.

¡Liberen a Islandia!

2012-05-15T21:27:00+00:00

En Islandia han decidido pasar todas sus administraciones públicas al software libre, dejando atrás así el software privatdo.Tryggvi Björgvinsson, líder del proyecto, afirmó que quieren así sentar una sólida base que sirva de ejemplo a otras instituciones. Así, ya cinco de las treinta y dos escuelas públicas “funcionan” completamente con software libre.

Fedora y su evolución

2012-05-15T20:56:00+00:00

Como los niños entenderían ¿Qué es Software Libre?

2012-05-15T20:44:00+00:00

OCaml Users and Developers conference 2012

2012-05-15T19:43:25+00:00

The OCaml Users and Developers Conference (14th September 2012, Copenhagen) is looking for speakers!

I’m intending to submit a talk on using OCaml for code generation in libguestfs.

Going to Fudcon KualaLumpur 2012

2012-05-15T17:21:28+00:00

Lots of back packing + back to blogging

Two sessions there + ongoing slides for that … and yes I am going to #fudconkl2012 .

See you all there

Less than 56 Hours to FUDCon Kuala Lumpur 2012

2012-05-15T16:55:27+00:00

Our swags and T-Shirt arrived today. With just less than 56 hours left to FUDCon Kuala Lumpur, we are starting to feel the heat.

We have sent out our guidebook yesterday to registered attendees (which reached 460 registrants today). For those how just registered and did not receive it, download it by clicking this link.

Swag pics:

lanyards(300units)

buttons (500units)

stickers (350units)

No T-shirt (170units) picture? .. well .. its in a huge thick plastic wrap and I do not want to open it yet as it'll be difficult to store afterwards.

Remember that swags are limited and are given by early register early & early arrive basis. The early 50 registrants on the website (excluding crew & volunteers) we will hold on to your swags for until 9:30AM, if you did not turn up by then, we'll give it away to those who arrived early.

We plan to open the registration booth on 8:30AM Friday.

See you on Friday!

- Izhar -

Back Door Async Replication

2012-05-15T15:15:41+00:00

My standing search for “glusterfs” on Twitter got me into an interesting discussion with Dr. Shawn Tan about an interesting GlusterFS configuration for several workstations. At first my reaction was panic, because I could see potential for data loss in that configuration, but “you’re using it wrong” is rarely a productive response from a developer. Dr. Tan’s use case is actually quite valid. It just happens to be one that GlusterFS doesn’t deal with very well right now, so instead of spending my time feeling superior I spent it thinking about what we could do differently instead. The basic parameters here are:

Provide shared access to data across several workstations in a cluster.
Use replication to ensure that data can survive a workstation failure.
Maximize the percentage of reads that are local.
Maximize the percentage of writes that are local.

The first two points are pretty clearly good reasons to use GlusterFS or something like it. It’s the third and particularly fourth points where this configuration runs into trouble. Caching data for reads is fine, as long as you understand your consistency needs and don’t give up more consistency than your applications or users can really tolerate. Caching (actually buffering) writes is much more dangerous. If your goal is to ensure that data can survive a workstation failure, then writing it only on the workstation and leaving it there forever is simply a non-starter. However, writing it initially on the workstation and writing it asynchronously to another machine might shrink the window of vulnerability to an acceptable level. GlusterFS’s “geosync” won’t do many-to-one replication, which would be very space-inefficient anyway, so how else might we get something like the same effect?

One part of the solution would be something like the “NUFA” (Non Uniform File Access) feature that GlusterFS used to have, but which has been deprecated since “DHT” replaced “unify” before I was even seriously involved with the project. The idea behind NUFA was to create files locally whenever possible, and write to those local files instead of via the “normal” kind of distribution. That sounds ideal except that as soon as you add replication you’re writing remotely again, so NUFA’s not doing anything for you . . . or is it? It’s making sure one copy of the file is local. What if we could make that the only copy initially, so that we only do local writes in the main I/O path, but with some control over how long it takes before that data does get replicated somewhere else? That’s where my bypass translator comes in. It’s not production quality yet, but it basically fits the bill of setting up two replicas and then populating only one. Since it works with the regular AFR translator, you can then use regular self-heal to propagate the changes to another replica. With self-heal becoming more precise, thanks to lots of hard work by other members of the GlusterFS team, this will soon be a pretty efficient process. The only thing you lose is strict ordering. Thus, you might not want to do this for parallel applications that depend on such ordering but it’s probably fine for sharing on the time scale of a person working on one workstation before lunch and a different one after. Your I/O flow would be something like this:

User at workstation A creates a new file and writes data into it. The file is actually created on A and B, but the data is only written into A (with all of the pending flags set so AFR knows that B needs an update).
Every ten minutes, a job runs that “self-heals” partially written files from A to B, A to C, or in any other case where A shares a replica set with one of the other workstations.
The same user subsequently sits down at workstation C and reads the same file. It’s read from A and/or B, which is unfortunate, but that’s a case where we can deploy caching to remove some of the sting.

If a user gets up and moves immediately to another workstation, they might see stale data. If they then start modifying the same files from the new workstation, split-brain problems could result. Similarly, if a failure happens on A before a new file has been propagated to B, then that file’s actual contents will be unavailable until A (or at least the relevant disk from A) comes back. These might be absolute show-stoppers for most people. On the other hand, I could see using something like this myself e.g. between my work machine, my home machine, one or more laptops, and my general-purpose server at Rackspace. On the third hand, maybe I’ll just wait until I have a chance to work on my personal Patch Based Filesystem project which would handle this exact use case even better.

As a further optimization, the system could detect reads and/or existing-file writes at C for files living on A and B, then start pro-actively relocating related files so that one copy is at C. Ideally, this would mean adding a replica at C (turning two-way replication into three-way) and then deleting the replica at A or B (going back to two-way) so that data protection is maintained throughout. Unfortunately, current GlusterFS replication doesn’t offer this kind of flexibility to change replication levels on a per-file or temporary basis. We don’t have the detection of changes to trigger this migration, or the policy to identify related files, so this is all clearly in the future anyway. However, it’s a direction we could go, step by incremental step, providing at least a little bit of additional value to at least a few users each time.

Gitolite3

2012-05-15T14:54:03+00:00

I’ve finished packaging Gitolite 3.01 up, and the review ishere. Interested parties have a look! My personal upgrade went very smoothly.

Instalar Adobe Muse en Linux [HowTo]

2012-05-15T14:23:47+00:00

Adobe Muse es una nueva herramienta de Adobe enfocada al Diseño web. Su finalidad es proveer a los diseñadores de una herramienta que permita la creación de sitios web usando las últimas tecnologías (HTML 5 & Friends) sin necesidad de escribir una sola línea de código. Se trata de algo novedoso puesto que anteriormente el proceso de creación de páginas web era un poco más tedioso: Diseñabas tu plantilla en Photoshop/Gimp y luego tenías que transformarla en código. Con Muse la cosa ya no es así, en un solo programa DISEÑAS y el Código aparece por si solo, no necesitas tocarlo. Es muy fácil de usar (muy a la drag&drop) y bastante útil, sin embargo, (como es de costumbre con los productos Adobe) sólo está disponible para Windows y MacOS, más no para Linux. Ésta limitante no importa gracias a Wine, pues podemos instalarlo en cualquier distro usando ésta famosa utilidad y la versión para Windows del programa, veamos a continuación como:

NOTA: Las instrucciones a continuación detallan el proceso a seguir para Ubuntu y Fedora, pero los mismos pasos aplican para cualquier otra distro linux cambiando los comandos utilizados donde aplique.

1) Descarga Adobe AIR y Muse

Necesitaremos ambos para lograr nuestro cometido. Descarga las versiones Windows de 32 Bits desde los siguientes enlaces:

AIR: http://get.adobe.com/es/air/otherversions/

Muse: http://www.adobe.com/cfusion/tdrc/index.cfm?product=muse&loc=en_us

2) Instala Wine + Winetricks de 32 Bits en tu distro

Sin importar el hecho de si usas un sistema de 32 o 64 bits, es importante que instales la versión de 32 bits de Wine forzosamente en tu sistema. Mientras que en sistemas de 32 Bits instalar Wine es tan fácil como hacer:

1 2	sudo apt-get install wine1.4 (Ubuntu 12.04) sudo yum -y install wine (Fedora)

En versiones de 64 Bits la cosa cambia. Para Fedora checa éste tuto y para Ubuntu instala con el comando de arriba y luego checa éste tuto. También necesitaremos a Winetricks. En Ubuntu se instala junto con Wine, Mientras que para fedora tienes que instalarlo como se explica acá.

3) Componentes extra

Necesitaremos algunos componentes extra para hacer que el programa funcione como debe, para instalarlos hacemos:

sudo apt-get install cabextract (Ubuntu)
sudo yum -y install cabextract (Fedora)
winetricks mfc42 vcrun2008 dotnet20 
winetricks wininet winhttp pngfilt
winetricks liberation corefonts

4) mscorefonts

Otra cosa que ocuparemos será las Fuentes Microsoft. En Ubuntu se instalan junto con los restricted extras mientras que para Fedora querrás checar éste tutorial.

5) Instalando!

Ahora sólo da click secundario en el ejecutable de Adobe AIR y selecciona "Abrir Con>Wine" Tardará un poco en iniciar. Una vez instalado, haz lo mismo con el ejecutable de Muse, igual tardará un poco pero a final de cuentas instalará. Asegúrate de marcar "Iniciar Aplicación después de instalar". Una vez instalado te pedirá que inicies sesión con un Adobe ID, si no tienes uno, entonces créatelo acá.

Éso es todo. Siguiendo éstos pasos podremos instalar Adobe Muse en nuestro Sistema Linux... Como verán, se trata de una versión de prueba que requiere una suscripción mensual o anual para dejarla funcionando después de 30 días (dependiendo lo que les agrade más), pero es bastante útil a decir verdad... Recordar también que ésta emulación puede presentar algunas fallas, por lo tanto, no lo usen como herramienta de producción al 100% sobre de Linux, quizá tendrán que diseñar unas cosas en Muse y luego hacer tweaks en el código generado para que todo quede personalizado como quieran.

Zeitgeist optimizations

2012-05-15T12:54:35+00:00

The Zeitgeist team has been hard at work lately. We recently moved from Launchpad and Bzr to freedesktop.org and git, just in time for the 0.9 release. Since then, Seif and I have been hammering away at making a bunch of speed improvements and trying to scale the engine to be able to handle a few billion events just as fast as it can handle a few hundred thousand.

Our first experiment has been focusing on the sqlite indexes. This one index tweak appears to increase our benchmark speeds by almost 45%:

CREATE INDEX event_timestamp_subj_interp_subj_id_id
                   ON event(timestamp, subj_interpretation, subj_id, id)

And here’s the graph to prove it:

Shorter lines are better. Blue is the 0.9 release, and the orange is the tiny index change. All of our raw data is in this google spreadsheet, so feel free to have a look. I can’t guarantee that you’ll immediately understand the information, but I’m open to explain it to those interested.

<g:plusone href="http://wm161.net/2012/05/15/zeitgeist-optimizations/" size="tall"></g:plusone>

Backup a symbian operated phone on Fedora

2012-05-15T12:44:56+00:00

Small hacking to backup a symbian operated phone on Fedora.

English version

If you have a symbian based phone there are several ways to backup your files.

If it is pictures and music, you can just mount your phone using a USB cable, browse the folders and retrieve your files. (Note that you can do the same in bluetooth of course).

If you want to backup your contacts, messages or calendar, then it is a little more tricky.

Nokia claims that symbian supports SyncML. Although, apparently some person managed to get it working on their phone, I did not manage on mine.

Then I found the project called Series-60-remote. This is a very nice tool providing a nice interface to do several things:

Backup all your contacts, agenda and messages into a small sqlite (or mysql) database
Manage your contact or message on the phone from your computer through this interface
Send and read your text messages directly from the interface (allowing a text-based 'chat').

Unfortunately for me, the maintainer got an android and is working on a large rewrite of the project porting it also to android. This sounds very promising and I certainly enjoy it if I get an android phone.

But right now, I still have a symbian phone and I still want to back it up.

The interface is nice but I have been looking for a small cli tool with which I could back up my phone more frequently.

So I rewrote a part of the connection logic, it is now independent from PyQt and works stand-alone. It still requires the phone apps to be installed on the phone and a couple of python library depending on the format with which you want to backup your files (vCard vs ldif) but, hey, it works :-)

There are still more work to be done though.

If you want to follow up on this let me know and check the git repo

Some stats on the Fedora-fr forum

2012-05-15T12:44:18+00:00

There are some statistics on the forums from Fedora-fr.org

English version (French version)

The other day I discovered that I had access from the administration pages of the Fedora-fr.org forums to some statistics from January 2002 to those days.

These stats show the number of messages, topics and members for each month.

So I did the graphs.

Evolution of the number of new members for each month. Evolution of the total number of members.

Evolution of the number of new messages for each month. Evolution of the total number of messages.

Evolution of the number of new topics for each month. Evolution of the total number of topics.

To note, the last entry point is biased/incomplete as November 2011 has not ended yet (and the stats are already a few days old).

It is also funny to see that the forums have been really popular and recruited new members with new question around 2007 :-)

Everything is available:

Ready! Get set!! Go!!!

2012-05-15T12:34:17+00:00

Well! It is upon us! Time to run!

Do not forget:

- Adapter converters

- The guidebook!

- Batteries

- Power spikes

- Your Fedora T-shirts!

- Stickers!

- Clean underwear!

I shall see you at the conference!

درایورهای گرافیکی جدید در کرنل ۳.۴

2012-05-15T11:15:29+00:00

کرنل ۳.۴ شامل داریورهای تراشه‌های ای‌ام‌دی، انویدیا و اینتل خواهد بود.

به گزارش اچ کرنل جدید که قرار است اواخر همین ماه منتشر شود همچنین شامل درایورهای جدید یو‌اس‌بی DisplayLink و بهبودهایی برای تراشه‌‌های ترکیبی اوپتیموس شرکت انویدیا باشد.

در همین زمینه:

Auto-Installing oVirt Node

2012-05-15T09:27:29+00:00

oVirt - maybe you've heard about it. It's a project to create an open IaaS "virtualization management system" - So a bit like OpenStack, but different.
Fedora is the base for oVirt's hypervisor: "Node". Basicaly this is a stripped down Fedora, enriched with a couple of packages to prvide just enough to host some virtual guests and do some basic configuration.

Personally I'd like to use Node in conjunction with Gnome Boxes or virt-manager. But this is currently not possible - but we might get closer to it when solving this bug.
Anyhow, to quickly install oVirt Node you just need to add two (or three) additional kernel arguments:

BOOTIF=ethX storage_init

You should/could also add

adminpw=$ADMINPW

ADMINPW=$(openssl passwd -salt SALT) is a salted password, so you can log in (as admin) after the installation. Alternatively you can boot into single mode to reset the password.

The parameters above install oVirt node without user intervention, setup networking on ethX and erase all data on the disk and create a defautl (lvm based) partitioning scheme.
The next step would be adding Node to oVirt Engine - or wait until it can be managed by virt-manager, which is much quicker to set-up :)

Tiny Core Linux 4.5

2012-05-15T07:40:24+00:00

Es ist schon eine Weile her seit ich das letzte Mal mit Tiny Core Linux gespielt habe. Nach der Installation startet Tiny Core Linux extrem schnell. Als Vergleich ein Start von Alpine Linux und einer von Tiny Core Linux in einer virtuellen Maschine.

Alpine Linux: 12 s (bis Login)
Tiny Core Linux: 6 s

Die Messung ist nicht ganz fair, denn eine Alpine Linux-Instalaltion ist etwas umfangreicher als die von Tiny Core Linux. Der Start von Fedora dauert aber in jedem Fall um einiges länger. Ausser ich verzichte auf den Start der grafischen Oberfläche...der Vollständigkeithalber muss ich mal die Zeit messen bis eine Minimalinstallation von Fedora bereit ist. Dem Gefühl nach wird so sich etwa in der Grössenordnung von Alpine Linux bewegen

Sehr schön ist, dass bei Start von Tiny Core Linux eine Auswahl mit Window-Manager zur Verfügung steht.

HTML Image Mapping

2012-05-14T21:41:53+00:00

One of the tasks I had to do in $DAYJOB consisted in displaying a Map of Mexico and being able to highlight each state on mouseover using HTML to do it.

Of course, there’s a dozen or so Online Image Mapping tools, but they require you click every single point of the state, and unlike USA whose states are often squares or very basic shapes, Mexico has none of that. It’s all weird and squiggly lines.

To solve this issue, I found this brilliant Gimp plugin. All I had to do is select the state by selecting by color, saving it to a channel and exporting it. The result, if I may say so, is pretty darned good. If you have issues with the color selection tool, remember you can tweak the treshold values for the color selection, so that it selects more or less similar colors to the one you chose.

I decided to license the file as Apache License 2.0, and the code’s available at Github for forking. I hope this’ll help fellow Mexicans, as I found absolutely no usable Maps of Mexico for HTML that worked in the crappiest of browsers (Internet Explorer) and didn’t require Flash.

If you’d like to make your own map, the Gimp HTML Img Map plugin page has an excellent tutorial, my recommendation when exporting this, however, is to set the treshold for smoothing to 0.

LinuxTag Tickets

2012-05-14T19:35:18+00:00

If anybody still needs free tickets for LinuxTag, please comment below.

Falls noch jemand Gratis-Tickets für den LinuxTag braucht, bitte unten kommentieren.

ripping the X server a new driver API

2012-05-14T17:05:58+00:00

So I've been slowly writing the hotplug support v3 in between all the real jobs I have to do.

[side note: When I started out on hotplug. one of my goals was to avoid changing the server API/ABI too much so I could continue side by side testing.]

how did I get to v3?
v0.1: was called dynerama it failed miserably and proved that using Xinerama as the plugging layer was a bad plan.
v1: was the first time I decided to use an impedance layer between some server objects and driver objects.
v2: was the a major rebase of v1.

v2 was trucking along nicely and I managed to get the design to the stage where PRIME offloading intel/nouveau worked, USB device hotplug with udl worked, and GPU switch worked between two drivers. However v2 duplicated a lot of code and invented a whole new set of API objects called DrvXRec, so DrvScreenRec, DrvPixmapRec, DrvGCRec etc, this lead me to looking at the pain of merging this into the drivers and the server, and my goals of avoiding changing the API/ABI was getting in my way.

So before starting v3 I decided to rework some of the server "APIs".

The X server has two main bodies of code, one called DIX, and one called DDX. The DIX (device independent X) code and the DDX (Device dependent X code). In the X.org tree the dix lives up in the top level dirs, and for X.org server the DDX lives in hw/xfree86. The main object with info about protocol screens and GPUs is called ScreenRec in the DDX and ScrnInfoRec in the DIX. These are stored in two arrays, screenInfo.screens in the DIX and xf86Screens in the DDX, when code wants to convert between these it can do one of a few things.

a) lookup by index, both structs have an index value, so to go from ScrnInfo to Screen you look at screenInfo.screens[scrninfo->scrnIndex] and other way is xf86Screens[screen->myNum]. This is like the I didn't try and make an API, I just exposed everything.

b) ScrnInfo has a ScreenPtr in it, so some code can do ScrnInfo->pScreen to get the pointer to the dix struct. But this pointer is initialised after a bunch of code is called, so you really can't guarantee this pointer is going to be useful for you.

c) XF86SCRNINFO uses the DIX private subsystem to lookup the Scrn in the Screen's privates. This is the least used and probably slowest method.

So also screenInfo.screens contains the protocol screens we exposed to the clients, so this array cannot really change or move around. So I'd like to add screeninfo.gpuscreens and xf86GPUScreens and not have drivers know which set of info they are working on, however (a) totally screws this idea, since the indices are always looked up directly in the global arrays.

Now lots of the Screen/ScrnInfo APIs exposed to the drivers pass an int index as the first parameter, the function in the driver then goes and looks up the global arrays.

So my first API changes introduce some standard conversion functions xf86ScreenToScrn and xf86ScrnToScreen, and converts a lot of the server to use those. Yay an API. The second set of changes then changes all of the index passing APIs to pass ScrnInfoPtr or ScreenPtr, so the drivers don't go poking into global arrays. Now this is a major API change, it will involve slightly messy code in drivers that want to work with both servers, but I can't see a nicer way to do it. I've done a compat header file that will hopefully allows to cover a lot of this stuff where we don't see it.

I've ono other API introduction on the list, Glyph Pictures are another global array indexed by screen index, I've gone and added an accessor function so that drivers don't use the index anymore to get at the array contents directly.

Once this stuff lands in the server, a team of people will go forward and port the drivers to the new APIs (who am I kidding).

A plea to developers

2012-05-14T16:31:29+00:00

As a developer myself, I still fail to understand why you’d ever want to do this, but I still see it way too often to not say again what I say regularly: Don’t. Bundle. Libraries. I don’t care if it’s C/C++, Java, PHP, Python, there is a correct and easy way to use libraries. When you bundle:

You add to your maintenance burden.
You make it difficult or impossible for some distributions to include your software.
You put yourself and your users at a disadvantage at a disadvantage for security and bug fixes.
Somewhere, a baby seal is clubbed.

So . . .don’t!

Thank you.

Mailing list web interface magic

2012-05-14T15:43:56+00:00

For a while now folks I know have been talking about how to reoutfit Mailman so it has a proper web front-end. The idea would be to provide additional features, make open source mailing lists friendly to web forum loving people, and keep hardcore email-only contributors able to participate in the same medium as free-wheeling web forum afficionados.

Máirín put up a series of posts that not only give great visual thoughts, but are some powerfully good ideas on providing something a community can really gain from using. I’m not going to try summarizing, I just encourage you to read at least these two:

… and go ahead and read 7,750 pixels of mailing list thread, which came first anyway.

One thing that really struck me about these ideas is that many of the features that rely upon posting history can be run against an existing archive of messages. That means upgrading Mailman to features like these means quickly gaining a view in to the history of your mailing list that is a sizeable part of the richness of the new features. All the ideas that rely upon keywords in posts, previous posting history and frequency and topics covered, all the reputation ideas, all that would be seeded with as many months or years of archives a mailing list has. That’s really cool!

I really look forward to seeing some of these kind of changes in Mailman. I’m a big fan of Mailman on the straight mailing list management side. But over the years I’ve seen the wider and wider divide between the users who prefer web forums and those who prefer mailing lists (many of whom are contributors who want to interact with other users, but not on a web forum.) These ideas could provide a great stitching of that divide, without forcing a big change on either party.

My simple IT support rules

2012-05-14T15:28:42+00:00

Don't be an asshole.
If someone is being an asshole to you, see #1.
Are they upset at something? Identify and fix. Repeat as required.
Assholes are infectious and easy to flare. Avoid them, unless #3.

Joining Mozilla

2012-05-14T14:49:27+00:00

I’m very excited to say that as of today I’m a Mozillian! I’ll be working with the fine folks on the WebAPI team. This is an entirely new set of technologies for me so I’ll be learning a tonne. I’m looking forward to meeting and working with all the excellent people in the Mozilla and broader web communities. I’ll be working out of the beautiful Toronto office.

As for Eclipse and Fedora stuff, I’m really looking forward to the release of Linux Tools 1.0 and the rest of Juno in June, 2012. Before then, Fedora 17 will come out and is going to be an outstanding release, especially from the Eclipse point of view. As I’ll be consuming Eclipse tools for my new work, I’ll be sure to file and fix bugs and spread the good word about the power of Eclipse for C++ developers. When I have some good HOWTO content for developing Mozilla stuff with Eclipse, I’ll post it here and be sure to tag it so Planet Eclipse picks it up.

With apologies to Neil Young: keep on rockin’ in the Free and Open Source world!

Tip: Using mount-local API from C

2012-05-14T11:22:36+00:00

Previously if you wanted to mount a disk image on the host from libguestfs, you had to use a separate program called guestmount.

A couple of months ago, we added FUSE support directly into the libguestfs API, and rewrote guestmount to use it. This also means you can use FUSE from your own libguestfs programs.

Yesterday I wrote a short example of using the mount-local API from C. This program creates a new disk image, formats it etc using libguestfs, then gives you a subshell so you can copy your own files, directories etc. in. When you quit the subshell, the disk image is synced and closed, and you end up with a virtual disk image containing all the files you just added. (Nothing that you couldn’t easily do before, but a nice little demonstration anyway).

Here it is in use:

$ gcc -Wall mount_local.c -o mount_local -lguestfs
$ ./mount_local /tmp/test.img

This is the 'mount-local' demonstration program.  Follow the
instructions on screen.

Creating and formatting the disk image, please wait a moment ...

The _current directory_ is a FUSE filesystem backed by the disk
image which is managed by libguestfs.  Any files or directories
you copy into here (up to 512 MB) will be saved into the disk
image.  You can also delete files, create certain special files
and so on.

When you have finished adding files, hit ^D or exit to exit the
shell and return to the mount-local program.

mount-local-shell> ls
lost+found  PUT_FILES_AND_DIRECTORIES_HERE

From the subshell, I copy in some files:

mount-local-shell> cp -a /usr/share/doc/libguestfs-devel-1.17.40/ .
mount-local-shell> ls
libguestfs-devel-1.17.40  lost+found  PUT_FILES_AND_DIRECTORIES_HERE
mount-local-shell> ls libguestfs-devel-1.17.40/
AUTHORS			       example-ubuntu.xml
BUGS			       example-windows-2003-x64-cd.xml
ChangeLog		       example-windows-2003-x86-cd.xml
copy_over.c		       example-windows.xml
create_disk.c		       example-windows-xp-cd.xml
display_icon.c		       HACKING
example-debian-netinst-cd.xml  inspect_vm.c
example-debian.xml	       README
example-fedora-dvd.xml	       RELEASE-NOTES
example-fedora-netinst-cd.xml  ROADMAP
example-fedora.xml	       TODO
example-rhel-6-dvd.xml	       virt-dhcp-address.c
example-rhel-6-netinst-cd.xml  virt-inspector.rng
example-ubuntu-live-cd.xml

After copying in my files, I exit from the subshell:

mount-local-shell> exit

Any files or directories that you copied in have been saved into
the disk image called '/tmp/test.img'.

Try opening the disk image with guestfish to see those files:

  guestfish -a /tmp/test.img -m /dev/sda1

Here is the disk image that was created (note it is sparse, so it’s not really so large as it appears):

$ ll /tmp/test.img
-rw-r--r--. 1 rjones rjones 536870912 May 14 12:03 /tmp/test.img
$ du -sh /tmp/test.img
18M	/tmp/test.img

We can use guestfish to look inside it:

$ guestfish -a /tmp/test.img -m /dev/sda1

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> ll /
total 18
drwxr-xr-x  4 root root  1024 May 14 12:03 .
drwxr-xr-x 23 1000 1000  4096 May 14 12:18 ..
-rw-r--r--  1 root root     0 May 14 12:03 PUT_FILES_AND_DIRECTORIES_HERE
drwxr-xr-x  2 root root  1024 May 14 08:37 libguestfs-devel-1.17.40
drwx------  2 root root 12288 May 14 12:03 lost+found

Download the test program here: https://github.com/libguestfs/libguestfs/blob/master/examples/mount_local.c

Public SCAP Contents

2012-05-14T11:15:40+00:00

As stated earlier, writing own SCAP content from scratch is rather hard. One needs to understand at least XCCDF, and often also OVAL layer. On the other hand, customization (tailoring) of existing content is easy. One needs only to review the existing and decide which checks he (or she) wants to include in his (her) security profile, and customize a few variables. There is even opensource graphical tool for tailoring, it is called scap-workbench.

There are groups publishing their XCCDF content under opensource licences. So anyone may start hacking on top of them.

USGCB for RHEL5 Desktop. The United States Government Configuration Baseline, It's the official SCAP content for desktops within federal agencies. It has been developed at NIST which collaborated with DoD and Red Hat, Inc. Unfortunatelly, it turned out, that the content tends to become out of date and some parts of OVAL may not fit nicely to the latest Fedoras.
SCAP Security Guide for RHEL6. It reassumes and extends USGCB, containing profiles for desktop, server, and ftp server. This content is under active community development.
SCE Community Content. This content does not use OVAL, instead it contains arbitrary scripts as checking engine. Arbitrary scripts might be no-go for some deployments, on the other hand they allow rapid development which might came handy elsewhere. The content combines rules from STIG, Aqueduct, and Sectool.
Content Based on Sectool and obsoleting it. It's also included in the previous.
OpenSCAP Content for Fedora 14. Exemplary SCAP content created for Fedora 14 and maintained by OpenSCAP community.
OpenSCAP Content for RHEL6.

Please let me know, in case you are aware about another group publishing their XCCDF content. I would like to have them added.

Fedora in pillole: come registrare il desktop?

2012-05-14T10:27:00+00:00

Se usate Fedora nella versione di default con Gnome-Shell e avete la necessità di produrre uno screencast, pochi sanno che basta utilizzare la combinazione di tasti:

CTRL + SHIFT + ALT + R

per avviare una registrazione in formato WebM di ciò che viene visualizzato sul vostro schermo.

La combinazione di tasti vale sia per l'avvio che per l'arresto della registrazione.

Alla fine troverete nella vostra home directory un file chiamato shell-data_odierna-#.webm(ad esempio, shell-20120514-1.webm per il primo screencast registrato oggi)

Provare per credere :-)

Fedora 17 發行派對 @ 臺中

2012-05-14T09:29:43+00:00

Fedora 17 即將在 5/22 發行，一如往常將帶給大家 Linux 界中最尖端的技術、最新版的自由軟體。

我們將於 6/3 (日) 在臺中舉辦發行派對，歡迎大家一同到場共襄盛舉！本活動由 Fedora 專案官方贊助，全程免費。转载自邮件列表

Fedora 是一個以 Linux 為基礎的作業系統，是一套能讓你電腦運作自如的軟體集合。

Fedora 專案是個國際性社群的名稱，熱愛自由軟體、使用自由軟體、也打造自由軟體。一直以來，Fedora 都是 Linux 最新自由軟體的試驗場，包括 ibus、packagekit、system-config-printer... 等都源自於此；此外 Fedora 也一直與上游緊密合作，讓不是使用 Fedora 的人也能一起受益。

主題將以 Fedora 為主，介紹 Fedora 專案、新版的特色 (例如 GNOME 3.4)、以及其他應用與分享等。

活動開始時間：2012/06/03 14:00
活動結束時間：2012/06/03 17:00
地點名稱：黑貓月亮咖啡館台灣台中市西屯區大墩二十街111號
活動人數上限：25

活動報名網頁

Freelance / Storyboard

2012-05-14T07:02:00+00:00

I am doing a lot of photography lately but I also didn't forget about graphics, the latest project I was involved lately was doing freelance work on a proprietary commercial project for creating storyboards). While is not Free software, it allows to leverage my experience from my old Fedora webcomic and even build upon some of the assets I developed as a follow-up. From a technology point of view, the project is cool as it uses SVG for its files, so once installed you can extend it easily with self-made graphics or with images from the Open Clip Art Library, for example.

Unfortunately the app is Windows-only, made with .NET, so I can't easily provide screenshots (I use Inkscape and Fedora for my part of the work), but here's an example image from the Storyboard That website (probably people will recognize the graphics style):

Upcoming FAmSCo elections and a special nomination

2012-05-14T01:34:34+00:00

As you probably have heard, FAmSCo elections are coming earlier this year. 7 seats will be up for next election and 3 up for the December election. The anticipated elections are a side effect of the FAmSCo election reforms made on the current term. Personally I’d prefer this transition to be made on December, when we usually have FAmSCo elections and when the current term was supposed to end initially. IMHO having elections on the middle of the current term means that the rule is being changed during the game. On previous elections the ambassadors voted for a full term, and my position is that – ideally – we should have fulfilled it in respect to the voters. On the other hand the majority of FAmSCo realize that there are more advantages than disadvantages in making the transition now, for instance the possibility of filling vacant seats.

Although I do not agree with the transition timing, the reform in FAmSCo elections is a great improvement. FAmSCo chair, Christoph Wickert, bravely conducted this effort and now more people will be involved in the elections and the next FAmSCo term hopefully will be better than the previous ones. Despite the initial polemic regarding the transition, the new rules in place are solid and are the result of a work made by people committed to future of our community.

I believe that this was a short but intense term for everyone in FAmSCo and for me it ends with the feeling of mission accomplished. The previous term, which I was also a part of, was completed and more devoted to small but numerous changes, in contrast with those few big changes promoted by the current term. After those two terms I feel like my mission as a FAmSCo member is now fulfilled and I’m not running for reelection again. Therefore I would like to nominate Daniel Bruno as a candidate for next FAmSCo elections. Daniel has been on the road with us for a long while now and has been an excellent mentor for LATAM. He also did a good job maintaining our local infrastructure at projetofedora.org as well as on building up the open source community in northern Brazil.

After the elections I’ll concentrate myself on organizing the Fedora participation at FISL and do some groundwork helping the Brazilian Portuguese Translation Team. The team needs to improve it’s documentation and put some translations back on shape. That’s what we are willing to do on our upcoming Translation FAD and since I have some experience on translation processes from my early days on Fedora I’ll be glad to help them.

liberate your cloud data

2012-05-13T21:18:03+00:00

Fosscomm just ended and this post is just a placeholder for my Saturday speech about available Free Software solutions out there that can be used for self-hosted cloud services. Let’s get control over our data :)

Full presentation: pdf

GSoC 2012: On-Demand Fedora Build Service: Update #3

2012-05-13T19:37:50+00:00

I have pushed a working snapshot of the image building code to github[1]. Here is a sample run of the code:
$ sudo python imagebuild.py -t boot -a i386 -o image_op1 -w image_work -p fedora -r 17 -v 1 -nvr 'anaconda-17.26-1.fc17' -bid '318281' '311809'

This command line spawns the build process of a Fedora 17 boot.iso with a number of extra packages (specified via their NVR or build IDs):
$ sudo python imagebuild.py -t boot -a i386 -o image_op1 -w image_work -p fedora -r 17 -v 1 -nvr 'anaconda-17.26-1.fc17' -bid '318281' '311809'

Downloading Extra Packages

Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
Building Boot ISO
checking for root privileges
checking yum base object
setting up build architecture
setting up build parameters
installing runtime packages
running runtime-install.tmpl

downloading packages
( 1/592) [100%] downloading GConf2-3.2.5-1.fc17.i686.rpm
( 2/592) [100%] downloading ModemManager-0.5.2.0-1.fc17.i686.rpm
( 3/592) [100%] downloading NetworkManager-0.9.4.0-7.git20120403.fc17.i686.rpm
( 4/592) [100%] downloading NetworkManager-glib-0.9.4.0-7.git20120403.fc17....
( 4/592) [100%] downloading NetworkManager-glib-0.9.4.0-7.git20120403.fc17....
.
.

Finally, a boot.iso is created. Next, I plan to inegrate package retrieval from Koji via other methods. And then, support for creating Live images and DVD images. And then, the REST API/Web based service.

Some of the scripts I was experimenting with to pull packages are here [2].

[1] https://github.com/amitsaha/gsoc2012_fbs/tree/master/image_builder
[2] https://github.com/amitsaha/gsoc2012_fbs/tree/master/scripts

Installing Fedora 17 on my EEEpc using a kickstart!

2012-05-13T13:45:03+00:00

I’ve been trying to do this for a few days now. Figuring out the correct kickstart options, and then making the kickstart availble was a little work. It wasn’t difficult, it just required some experimentation really.

Here’s how I went about it:

The kickstart file looks like this (I install KDE on my eeepc to keep in touch with it. I use gnome on my main system):

#version=DEVEL
install
lang en_US.UTF-8
keyboard us
network –onboot no –device p1p2 –bootproto dhcp –noipv6 –hostname guest.pc
network –device wlan0 –noipv4 –noipv6 –hostname guest.pc
timezone Asia/Kolkata
rootpw –iscrypted foo-bar
selinux –enforcing
authconfig –enableshadow –passalgo=sha512
firewall –service=ssh
ignoredisk –drives=sdb,sdc
clearpart –all –drives=sda
autopart –type=lvm

firstboot –enabled

bootloader –location=mbr –timeout=5 –driveorder=sda –append=”rhgb quiet”

%packages
@core
@KDE Software Compilation
@X Window System
firefox
firstboot
vim*
%end

reboot

I used two pen drives. One for the installer, and another to provide the kickstart. Get the UUID of your pen drive using the ‘blkid’ command. This will be required.

To make the kickstart available to your eeepc during installation, hit ESCAPE when you get to the installer screen. You’ll be dropped to a boot: prompt.

linux ks=hd:UUID=:/

Tha’ts all really. Sit back, or go take a walk. When you get back, you have Fedora ready to use!

The page on the wiki with more information is here: http://fedoraproject.org/wiki/Anaconda/Kickstart

Oh! system-config-kickstart appears to be broken. Here’s the bug I filed.

Grazer LinuxTage 2012

2012-05-13T13:23:41+00:00

I have to admit: I love to be an engineer, to dive into issues and forget the time and everything around me until a problem is solved. This is good for the people having the technical problems which I solve then.

Now I learned why this is not good for organizing booths for free software events - alone. Not getting around to order posters or rollups for a booth in time lead to a strange presence of Fedora on Grazer Linuxtage. :)

You most likely know the sticker I used to improve the Austrian Tux. I had just those small things like buttons, stickers and DVDs with me in Graz, so I had to be a bit more creative...

The booth, however, was small but fine and quite well visited. I didn't expect that much of visitors before. The stories I heard of the event did let me think this will be a small event, so it was an awesome experience to see that I didn't had any of the Fedora Media, Christoph sent me, around, already at the half of the day. The other half didn't get boring because of that though. There still were a lot of Fedora Fanboys taking stickers and buttons with them and visitors asking the "usual" questions about the relationships between Fedora and RedHat and when the next release will come up and so on. Not to forget the jokers, who were asking for the pretended confusion between the Fedora and Facebook logos, of course...

A tip for any other Fedora Ambassador who is going to supervise a booth on an event some time: I think one reason that I had that much visitors at the booth without having big posters and such stuff around was, that I actually was *standing* next to or before the booth without a notebook. It is important to show that you are there for answering questions, not for hacking.

It was a good decision to get into an airplane for the second time in my life, to travel Hamburg-Frankfurt-Graz-Munich-Hamburg on just one weekend, even if I'd expected that there are Austrian Fedora Ambassadors as well who maybe are interested in supporting a Fedora Booth in Graz.

The weather in Graz was just great! 27°C, blue sky and sunshine all around.I got riled about myself because I missed to ask for vacation for the Monday after. The Tuesday after was a holiday in Germany anyway, so this could have gotten a really long weekend in Graz.

Beneath a quite nice view to the Alps from above when getting there, it was also a pleasure to meet the free software family again after a long time as well as a lot of new free software contributors and interesting people on the event.

It also was interesting to learn about the Austrian organizations who care about political issues concerning free software and data retention stuff and so on. Now I understand a bit better what it means that a lot of these topics are European topics.

Since I already arrived there on Friday afternoon and was flying back on Sunday, there was a lot of time around the actual event on Saturday to get in touch with the people. On Friday I was helping out with preparations for the event and was lucky to see that the organizators had table cloth for *every*
project booth (thanks for your efforts anyway, gnokii ;) ).

Saturday morning when I went to the event I already met Michael, who was taking care of the LPI booth which was right between the boothes of Debian and Fedora. To the left of the Fedora booth there were the guys of Gentoo and one distribution I didn't knew of before: OpenArtist is intended to be used by musicians and graphic designers. Interesting that the whole distro is chiefly maintained by just one person.

On Saturday evening a social event took place where exhibitors and organizators where invited. You have to know that the creators of GRML are member of the organization team and Zsh
geeks as well. You can imagine this was a very nice evening if you're a bit nerdy. ;)

On the sunday morning after we met again for a brunch with the people who organized the event. The restaurant had a really big backyard, so we could enjoy the coffee out of the doors in the green with the nice weather. A bummer that I had to leave that early to get my flight back to Hamburg with
a longer stop in Munich because the captain pilot was in the cantine and forgot that he already should sit in an airplane...

P.S. For the same reason I was introducing this post with, this report actually took almost two more weeks to actually get written down. ;)

El viejo cursor de Fedora...

2012-05-13T10:10:34+00:00

Echaba de menos la apariencia del puntero de Fedora 8,9... aquel que cuando lanzábamos una aplicación se movía justo en la parte inferior derecha del cursos como unas bolitas, en las que nos mostraba que estaba cargando el o los programas, o bien cuando navegábamos por Internet con el "browser", y bueno, era parte de un diseño propio de Fedora que podíamos decir, la caracterizaba un poquito más, en cuanto diseño gráfico, que está muy bien cuidado por cierto, y con grandes diseños de escritorio como fondos de pantalla, los temas Splash cuando iniciamos sesión en nuestro Fedora, el tema de KDE plasma y mucho más que nos brinda el equipo de diseño gráfico de Fedora.

Seguro que, la mayor parte de nosotros, que hemos usado Fedora en el pasado, o los que llevan usándola desde versiones anteriores nos hemos dado cuenta de que el tema del puntero cambió a otro llamado “Adwaita”, que es fácilmente encontrable como paquete con el nombre de “adwaita-cursor-theme”.

Sin embargo, si queremos volver a usar aquel cursor que cuando cargaba un programa nos hipnotizaba la mirada con aquellos circulitos danzando a los segundos del reloj esperando a la carga de los programas, solo tenemos que instalar el paquete “bluecurve-cursor-theme”, cambiarlo en el menú de nuestro entorno de escritorio y listo.

$ su -c "yum install bluecurve-cursor-theme"

Cambiamos la fecha de Fedora Show

2012-05-13T10:06:35+00:00

Por x motivos hemos tenido que retrasar Fedora Show para el día 26 de Julio y sólo se realizará en horario de mañana, más concretamente de 11:00 a 13:00, en el salón de actos de la Biblioteca Insular de Arrecife de Lanzarote. No obstante, hemos decidido tomar prestada el día 27 a la misma hora, de 11:00 a 13:00 en la sala de telecomunicaciones de la primera planta también en la Biblioteca Insular de Arrecife de Lanzarote, para hacer una "Fedora Installation party", en español fiesta de instalación de Fedora, para que todos los que quieran instalarlo puedan hacerlo aquí, además de tener asesoramiento, responder dudas... los aconsejamos que para la fiesta de instalación vengan con los estómagos vacíos, porque habrá refrescos, bocadillos.... ¡para llenarlos! por cortesía del equipo de Fedora EMEA.

¡Todos estáis invitados!

Nota: Si vives en el extranjero y te gustaría venir a la presentación así como para pasar unos días, podemos facilitarte la documentación requerida por la Policia para que puedas asistir. (Carta de presentación)

Bridged networking notes with systemctl

2012-05-13T10:05:45+00:00

Bridged networking gives the ability to make the virtual machines appear as just any other machine on the LAN, and I find it very useful while managing virtual networks. From Fedora 15 on wards, systemd has become the default init system on Fedora. Here is some quick notes on configuring bridging using the systemctl utility to control services on the system:

Disable the NetworkManager service, and turn on the classic network service:

[root@tbox ~]# systemctl disable NetworkManager.service

[root@tbox ~]# systemctl enable network.service

Create a bridge interface file, and modify your em1 interface(Note that, since ‘Consistent Network Device Naming’ feature is introduced, netowkr interface names have changed to em[1234] (correspondig to physical locations) from ethX

 #----------------------------------------------------------#
[root@tbox network-scripts]# cat ifcfg-em1
DEVICE=em1
BOOTPROTO=dhcp
TYPE=Ethernet
ONBOOT=yes
IPV6INIT=no
NM_CONTROLLED=no
USERCTL=no
PEERDNS=yes
PEERROUTES=yes
BRIDGE=br0
#----------------------------------------------------------#
[root@tbox network-scripts]# cat ifcfg-br0 
DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0
[root@tbox network-scripts]# 
#----------------------------------------------------------#

Restart the network service:

 
[root@tbox ~]# systemctl restart network.service

Ensure ‘network’ service is enabled(and NetworkManager is disabled) for the next boot:

#----------------------------------------------------------#
[root@tbox ~]# systemctl is-enabled network.service
network.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig network --level=5
enabled
[root@tbox ~]#
#----------------------------------------------------------#
[root@tbox ~]# systemctl | grep -i network.service
network.service           loaded active running       LSB: Bring up/down networking
[root@tbox ~]# 
#----------------------------------------------------------#
[root@tbox ~]# systemctl is-enabled NetworkManager.service
disabled
[root@tbox ~]#
#----------------------------------------------------------#

End result — Bridge(br0) inteface will have the IP address, and the em1 would lose it, as expected:

#----------------------------------------------------------#
[root@tbox ~]# ifconfig br0
br0       Link encap:Ethernet  HWaddr 00:21:9B:73:E2:65  
          inet addr:ww.xx.yy.zz  Bcast:ww.xx.yy.255  Mask:255.255.255.0
          inet6 addr: 2620:52:0:41c9:221:9bff:fe73:e265/64 Scope:Global
          inet6 addr: fe80::44c0:2bff:fe0d:b38c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:26475163 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3031754 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1836168879 (1.7 GiB)  TX bytes:304657564 (290.5 MiB)

[root@tbox ~]# 
#----------------------------------------------------------#
[root@tbox ~]# ifconfig em1
em1       Link encap:Ethernet  HWaddr 00:21:9B:73:E2:65  
          inet6 addr: fe80::221:9bff:fe73:e265/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:28841542 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4659974 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3999390382 (3.7 GiB)  TX bytes:580866999 (553.9 MiB)
          Interrupt:16 
#----------------------------------------------------------#

(NOTE: I just masked out IP with ‘ww.xx.yy.zz’)

UPDATE: I totally forgot to discuss the recent additions like ‘iface-bridge/iface-unbridge’ to libvirt to simplify creating a bridge. If someone wants to try, this should be the sequence to create a bridge:
# virsh iface-begin
# virsh iface-bridge em1 br0
# virsh iface-commit

QElectroTech on the road to 0.3

2012-05-13T08:07:00+00:00

RPM of QElectroTech version 0.3a, an application to design electric diagrams, are available in remi-test for Fedora and Enterprise Linux.

While the version 0.22, available in the official repository, is quite old (more than 2 years), the project is working on a new major version of their electric diagrams editor. Official web site : http://qelectrotech.org/. Changes : Changelog from 0.22 to 0.3a Installation by YUM : yum --enablerepo=remi-test install qelectrotech RPM (version... Lire QElectroTech on the road to 0.3

Tzinga – An energy drink startup in India

2012-05-13T07:52:58+00:00

An energy drink startup in India.

In, India we mostly consume only areated drinks. Energy drinks are kind of very rare to see if you consider both urban and rural landscape. So far I have come across a very few engergy drinks – RedBull, Tzinga, Could9, Rio etc. A friend of mine asked me to try this engergy drink called Tzinga.

Tzinga drinks

I received a package with all the three flavors ( shown in the picture above ). What’s my take on this?

The cost factor: The cost is really very cheap, at only Rs. 20, per packet.
The kick: All of these contain caffeine equivalent to 1 cup of coffee.

The three flavors one by one, in the order I liked them.
LEMON MINT: This is the best flavor in my opinion, very smooth and bit tangy. I and my friends liked it.
TROPICAL TRIP: This again is a very smooth flavor. I and my friends liked it.
MANGO STRAWBERRY: This one I didn’t like as it tasted a little bit bitter.

In sum, I would recommend Tzinga to all the energy drink fans.