This morning, at the 8AM (Pacific) OpenSolaris Governing Board (OGB) meeting, the following was proposed and unanimously resolved:

The OGB is keen to promote the uptake and open development of OpenSolaris and to work on behalf of the community with Oracle, as such the OGB needs Oracle to appoint a liaison by August 16, 2010, who has the the authority to talk about the future of OpenSolaris and its interaction with the OpenSolaris community otherwise the OGB will take action at the August 23 meeting to trigger the clause in the OGB charter that will return control of the community to Oracle.

That is to say, "start talking to us or we'll just shot ourselves in the head."

I made my opinion very clear via the IRC back-channel during the call. At least my call for a liaison was added into the resolution, but I am extremely opposed to this cowardly act.

What exactly do we have to gain or Oracle to loose? All Oracle does is runs out the clock, the entire OGB resigns, and then the one little bit of control the community has is gone. What motive, other than a benevolent act to garner press attention, does Oracle have to comply? We've just made their job easier.

I once advocated this kind of self-implosion tactic back in the Sun days. The reason was to re-organize the OpenSolaris leadership to be more engaged and industry focused. That was a good idea back in the days when I had faith that Sun would "do the right thing". However, those times have past. Oracle has made it clear that it either controls things or it doesn't... there is no give and take. I don't think we can demolish the structure and believe that Oracle will re-organize in such a way as to give the community more power. It was a long shot with Sun anyway.

Frankly, imho, this is just the OGB throwing its hands in the air. The body has been useless for a long time, but only because it has chosen to be. The majority of the OGB's life its wasted by trying to restrict its own authority by endlessly debating and re-writting the constitution. Its never lead anything, and it isn't now.

But the fact that its a wet rag doesn't mean we should simply throw in the towel. A weak seat of power is better than no seat at the table.

So where do we go from here? Who knows. At this point the die is cast and OGB is putting up their last stand. Maybe Oracle gets serious and does something, but I really doubt it. Not because they can't, but because its not in their best interest. Why kill something intent on killing itself.

My only concern as this point is to not loose regular code updates and access to the bug database. Yes, the existing code is "out there", but Oracle is still the biggest contributor, 99.999 to 1. Anyone can fork at any time right now, as is, so if your going to do that why would you risk cutting off the huge contributions continuously made by Oracle?

We're in no worse a position right now than we were during the Sun days. They didn't communicate, we had no visibility or impact on the OpenSolaris distribution, etc. Don't fall into the lie that things are now "worse" than they were... they aren't. Its status quo. The difference is that the OGB is no longer composed of Sun insiders who can get a sense of control from hallway conversations and are now as blind and weak as those of us in the community always have been.

The request for a liaison is a good one... I support it. But damnit, put the gun down. We don't need to act like irrational children having a tantrum. Ultimatums rarely workout the way you hope.

The bar is lower than the original resolution was, so we'll hope for the best and see.

> Read More... | Digg This!

If you didn't see it earlier this week in Robert Milkowski's blog, Oracle has provided an updated roadmap.

Simply it states the following:

• Oracle will continue to make OpenSolaris available as open source and Oracle will continue to actively support and participate in the OpenSolaris community
• Oracle is investing more in Solaris than Sun did prior to the acquisition, and will continue to contribute innovative technologies to OpenSolaris, as Oracle already does for many other open source projects

  On the "Solaris Near Term Roadmap", they state that Solaris 10 Update 9 will come some time in 2010 focusing on platform support and Oracle product integrations.

  OpenSolaris is slated for an update in the "1st half CY2010". It will be based on snv_134 and be called OpenSolaris 2010.??.

  No surprises here. But given the lack of information, no surprises isn't so bad.

  This does cause me to think I might have been wrong in my belief that Solaris 11 would arrive this year. I still think it's not a bad assumption, but if it doesn't come I'll be really curious as to what the heck Solaris Engineering is so busy working on.

  
  
  > Read More... | Digg This!

My photos from Tokyo BarCamp 2010. 

> Read More... | Digg This!

I attended the Oracle IOUC conference call last night, and I'm trying to ask some questions and get some conversation going on list. See my comments throughout the growing thread on advocacy-discuss and osug-leaders.

Personally, I am not on the UG team at Oracle. I'm still on the original OpenSolaris team on which I started in 2004. So, I don't have any inside information about any of these new programs. I'm just trying to help facilitate any transitions going on. Also, I've been directly involved in the OSUGs since the beginning of OpenSolaris, and I'm proud of what we've accomplished over the years -- when many people out there said we couldn't do it. But change is clearly here. And I strongly encourage OpenSolaris community members to fully engage in these conference calls verbally and give feedback in writing on list. I subscribed the Oracle UG team mailing list to advocacy-discuss and osug-leaders, so posting there is fine. The Oracle team will see your comments. And they are directly asking for your comments. The time is now, guys. There is now a direct connection for the OpenSolaris User Groups to Oracle. Take it. See this as an opportunity to build on what we've done before, and perhaps to move in a new direction. Get in there and pitch your stuff. That's the only way we can educate Oracle about what we've done, and it's the only way we can learn about what they do and how that might benefit us. Jeb Dasteel from Oracle was clear on the call that this program will evolve over the next couple of quarters, so the opportunity to engage Oracle about user group issues is here. Right now.



> Read More... | Digg This!

I have tried so hard to be positive and encourage people... but I must bring sad news. The first of all OpenSolaris User Groups, the Silicon Valley OpenSolaris Users Group, may be ending, as we know it, at its final meeting in a Sun campus this Thursday, May 27th.

According to SVOSUG leader, the lovely Alta Elstad, Oracle policy does not permit user groups to be run by Oracle employees or to meet on Oracle properties. Therefore, this next meeting is likely to be the last meeting in the wonderful Santa Clara Campus Agnews Mansion.

At this meeting there will be discussion about the potential future of the group, in some form or another.

If you are in the Silicon Valley please attend! Even if only to pay your respects. Meeting is at 7:30PM.

I want to pay a special honor to Alan DuBoff who started the group and grew it so successfully. I remember approaching him in 2005 I think, with the idea for a Silicon Valley User Group... instead of pitching my idea, he shared his ideas on such a group and had already started to put it together... so at least I got to be one of the first members. :) Alan's done a lot of great things for the community and we owe him a debt.

Alta Elstad took over leadership of the group when Alan left Sun (ask Alan, he has great stories, but I'll leave it at that) and has done a great job of keeping the group going, even though attendance has continued to decline. Alta is a remarkable woman and someone whom I have great esteem for. She is truly one of a kind.

Finally, to all those who've presented, participated with, and attended SVOSUG over the years, props to you all. I really hope that SVOSUG or some form of a Sun Users Group perhaps can continue onwards in the Silicon Valley.

> Read More... | Digg This!

I've discussed this many times before, but the reality still alludes many in our community. Therefore let me speak plainly such that you may understand.

OpenSolaris is not an open source project in the traditional sense. It can not and should not be thought of as such. If you attempt to do so you will only be frustrated and confused.

We have today seen yet another example of such a misunderstanding, when a community member wished to have a community leader punished by the OGB for failing (in his mind) to appropriately act in the best spirit of open source ideals. The specifics of the incident are irrelevant.

"OpenSolaris" as an "open source community" is little more than scaffolding around Solaris Engineering. There is a community structure, but its not really used. There is a governance and governing board, but they have no power and do nothing of significance. You can't put back (ie: commit) to ON directly and must sign a contrib agreement to even submit code, which is only actually commited at the sole discretion of Sun/Oracle employees (typically your assigned "sponsor").

Consider my analogy: scaffolding. Scaffolding allows you to navigate the exterior of a structure, to look in to the exterior windows on each level, to build things onto the exterior, or repair cracks and such, so on and so forth. Scaffolding doesn't allow you to climb through the windows. You can knock on the glass and point at things, but you aren't inside that structure... your on the outside, looking in.

So it is with OpenSolaris. Oracle owns the structure. We can't see into its inner rooms, only those with windows to the exterior. We have the ability to look in, suggest or propose or create changes or additions, but we can only hand them to others who actually have access. We can repair the outside of the structure or create custom additions, but no one on the inside of that structure needs to be involved, since its no part of the primary structure itself. We can chat with other folks on the scaffolding, share ideas and fellowship, but we're still on the outside. Those on the inside may choose to open a window to communicate with us, but only if they choose to do so.

Compare this with what you hear people say about OpenSolaris. I'll include myself. We like to think we're inside that building, along side everyone else. And we're not. We are only as empowered as the good employees of Oracle choose to let us be, through their sponsorship. And this is never guaranteed anywhere... its purely based on their good will.

If you consider the constitution of OpenSolaris, it defines within it a social organization. The conventional word would be: club. And that's what OpenSolaris is... a club. Little more, little less.

Whats important to you, my faithful reader, is to embrace the reality of this and then leverage it. If you dilute yourself, as I did for many years, into believing that you have some power or ability beyond influence, you will simply frustrate yourself and become ineffective.

What OpenSolaris provides is an amazing amount of access. The developers are available to you (CG/Project forums), the code is available to you, parts of the decision making process are available to you (ie: ARC, CR's, etc), the support structure is available to you (bug databases), and more. You are not an Oracle employee and you are not in Solaris Engineering... but you can hang out with the folks who are. You can help them, you can talk to them, you can influence them, you can even send code their way. At the end of the day, they are going to do what they and their superiors (now that they have them, har har), instruct them to... but a little constructive and friendly influence can go a long long way in this world.

The harsh reality is, that if you don't like this, you need to consider your options. Many of us have fought hard against it already and failed again and again and again, and only made enemies along the way. If you don't like it, consider starting your own project based on the code. Build onto that structure all you wish. Nexenta has done it very successfully, so has Belenix, and Blastwave too. Those projects are successful and strong, but they don't penetrate into the structure, that is, into Solaris Engineering. They stand independently.

One of the advantages of IPS will be that once a repo is added to your publishers list, you no longer thing about where the package comes from. Blastwave has traditionally provided add on packages in /opt/csw... clearly seperated from the OS. In the future, they will overlay the OS seamlessly. Therefore, consider a situation, such as above, in which the Sun version of some package isn't what you wish it to be? Simply create your own version, publish it on your repo, and then go tell the community that a better version is available, please add it to their publisher list and install it. Viva la revolution. This, is the way to effect change, far more effectively than pounding and yelling through the glass.

I really have a desire to see the anger, bitterness, and cynicism die away from our great community. Consider the venerable Sun-Net-Mangers list... perhaps the best support list ever. Resources such as this have existed in the Solaris community for almost two decades, without Sun/Oracle involvement. We, as a community, have a lot of capability with or without Sun/Oracle. Lets not loose that can-do spirit. Re-kindle that passion, leverage the OpenSolaris community for what it really is, and make good things happen.

> Read More... | Digg This!

Images from the May 2010 meeting of the Tokyo OpenSolaris Study Group. 

> Read More... | Digg This!

So IPS is here to stay, most likely, and you no doubt want to get a feel for things. So lets quickly create our first repository and package.

Hold on just one second! Lets get this clear first, IPS is all about network package repositories. There is no "local package" file format. Therefore, its a little confusing to say "create a package". So lets be crystal clear: we are going to send a bunch of files and meta-data as a package transaction to a package depot server. Fundamental concept.... moving on.

First things first, you can start an IPS Depot ("repository server", but the daemon is actually pkg.depotd) using the pkg/server SMF service, or start it manually in a shell (so you can watch it work, useful for your first time) like so:

root@quatro ~$  /usr/lib/pkg.depotd -d /export/pkg -p 80 --set-property publisher.prefix=cuddletech
[21/May/2010:00:40:19] INDEX Search Available
[21/May/2010:00:40:19] ENGINE Listening for SIGHUP.
[21/May/2010:00:40:19] ENGINE Listening for SIGTERM.
[21/May/2010:00:40:19] ENGINE Listening for SIGUSR1.
[21/May/2010:00:40:19] ENGINE Bus STARTING
[21/May/2010:00:40:19] ENGINE Started monitor thread '_TimeoutMonitor'.
[21/May/2010:00:40:19] ENGINE Serving on 0.0.0.0:80
[21/May/2010:00:40:19] ENGINE Bus STARTED
..

If you point your browser at http://localhost/ you'll see a web interface that looks identical to pkg.opensolaris.org. It has 0 packages. So lets add one.

There are many ways to add a package. The most popular way is to create a SysV package and then "convert it"... however I believe that to be extremely hypocritical, given that IPS is all about putting SysV packages to death. So we're going to avoid that method and assume that we're going to compile something from source and then package it.

I've decided to create a package for rsyslog. So I download the latest source and start to build it:

benr@quatro rsyslog-5.5.5$ ./configure --prefix=/usr
...
benr@quatro rsyslog-5.5.5$ gmake
...
benr@quatro rsyslog-5.5.5$ mkdir -p /tmp/staging_root
benr@quatro rsyslog-5.5.5$ DESTDIR=/tmp/staging_root gmake install
.....

Notice here that I built it to install into /usr, but before installing it I did a little switcharoo by over-riding the DESTDIR variable. The result is that gmake install installs into /tmp/staging_root, with all the links and configs as though it was in /usr.

Now that I've got my faked out installation, lets use pkgsend to import that directory structure:

benr@quatro rsyslog-5.5.5$ pkgsend open rsyslog@5.5.6
export PKG_TRANS_ID=1274430097_pkg%3A%2F%2Fcuddletech%2Frsyslog%405.5.6%2C5.11%3A20100521T082137Z
benr@quatro rsyslog-5.5.5$ export PKG_TRANS_ID=1274430097_pkg%3A%2F%2Fcuddletech%2Frsyslog%405.5.6%2C5.11%3A20100521T082137Z

benr@quatro rsyslog-5.5.5$ pkgsend import /tmp/staging_root

benr@quatro rsyslog-5.5.5$ pkgsend close
PUBLISHED
pkg://cuddletech/rsyslog@5.5.6,5.11:20100521T082137Z

And thats it. I first used pkgsend to open a new transaction to the depot server. This starts a new package. It returns to me a transaction ID which I export as a environmental variable used by future commands. Now I import the staging_root that I installed our tool into. Once done, I just close the transaction, and its done.

If I wanted to add meta-data, I could insert the following before closing the transaction:

$ pkgsend add set name=pkg.summary value="Rsyslog is an enhanced multi-threaded syslogd with a focus on security and reliability"
$ pkgsend add set name=pkg.description value="Rsyslog is an enhanced multi-threaded syslogd with a focus on security and reliability"
$ pkgsend add set name=description value="Rsyslog is an enhanced multi-threaded syslogd with a focus on security and reliability"
$ pkgsend add set name=packager value="Ben Rockwood (benr@cuddletech.com)"

Now go look at the repo in your browser again. Sure enough its there.

To install the new package, add your local repo to your IPS repo list and then install it:

benr@quatro ~$ pfexec pkg set-publisher -g http://localhost/ cuddletech
benr@quatro ~$ pkg publisher
PUBLISHER                             TYPE     STATUS   URI
opensolaris.org          (preferred)  origin   online   http://pkg.opensolaris.org/dev/
contrib.opensolaris.org               origin   online   http://pkg.opensolaris.org/contrib/
cuddletech                            origin   online   http://localhost/

benr@quatro ~$ pfexec pkg install rsyslog


DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1       33/33      0.7/0.7

PHASE                                        ACTIONS
Install Phase                                  41/41 

Pretty easy right? Creating packages can be really hard if you add all the files manually and nit-pick it, but personally I find this "import /some/dir" to work just fine. It also works on tar files! :)

Good luck with IPS.

> Read More... | Digg This!

> Read More... | Digg This!

For those of you who may be unclear on what BFU is, it has become short for Blazing Fast Upgrades. The Solaris ("Nevada OS/Network Consolidation", to be specific) source traditionally had the ability to output BFU images, where are really just cpio archives which can be overlaid on an installed system. The idea was to allow developers to quickly and painlessly (as possible) upgrade to the new builds without the usual patch hassles or doing a full re-install.

For some time now you've had the option to download CD/DVD images of Solaris. the source and tools themselves, or BFU images. When you combine BFU with the ACR (Automatic Conflict Resolution) tool, you could keep ugprading from build to build for a while before the works got so gummed up that you needed to reinstall.

You'll notice that snv_139 and snv_140 didn't include BFU's, making me wonder if this was a temporary problem or a permanent change.

Turns out BFU's are, essentially, gone for good. The source, supposedly, can still produce them, but they will no longer be provided with the source releases and very soon they won't be possible at all.

In March, Liane Praza announced in a Flag Day report, that among other things, that:

"BFU is still available (though ACR is not), but pkg image-update is the preferred testing method. Support for BFU is expected to be removed in a few builds."

That was March, "in a few builds" is now.

I'm informed by James McPherson that following the integration of the IPS packaging changes, the onu ("Os/Net Update") tool will be replacing BFU, preforming essentially the same function.

If you want to keep up on all the deep changes that are happening to ON because of IPS, please subscribe to the on-ips-dev mailing list.

I hope you can start to understand just how major a change IPS really is... its not a topical cream for Solaris... its a heart transplant.

> Read More... | Digg This!

Oracle today posted mail to two OpenSolaris mailing lists -- advocacy-discuss and osug-leaders -- outlining details for the new International Oracle User Group Community. With the acquisition of Sun, Oracle more than doubled the number of user groups in the program so the total is now 900. The program has multiple models as well. More details will follow in a series of open conference calls. See the link above for con-call numbers, dates, and times.


> Read More... | Digg This!

Here are some recent OpenSolaris and Linux links over the last few months: Some May Community Events in Japan, BarCamp Tokyo ????????, Tokyo OpenSolaris Study Group 2010.04, Work Speaks Louder than Words, Building Communities: Update, Tokyo Linux User Group 041610, OpenSolaris Night Seminar 041610, Tokyo Linux User Group 041010, Sun Japan, OpenSolaris & Linux: April in Tokyo, Updating Website Schedules: April, OpenSolaris DTrace @ Yokohama Linux UG, DTrace Day Tokyo: Photos, OpenSolaris IPS Tokyo 031910, Kato, DTrace, ZFS, Tokyo Linux User Group 031310

> Read More... | Digg This!

A few shots from the Tokyo OpenSolaris Study Group earlier today. About 40 people came by on a nice spring Saturday for the six sessions (three for administrators and three for developers).

> Read More... | Digg This!

If you've seen one of my talks on ZFS Tuning you've witnessed my love for the fsinfo provider (and the fsstat command as well) first hand. This amazing but poorly documented provider can give you some great high level insight into your I/O pattern. If you've ever used truss just to watch iops, then give this a whirl instead.

There are probes for each VFS operation:

   ID   PROVIDER            MODULE                          FUNCTION NAME
14937     fsinfo           genunix                       fop_vnevent vnevent
14938     fsinfo           genunix                       fop_shrlock shrlock
14939     fsinfo           genunix                    fop_getsecattr getsecattr
14940     fsinfo           genunix                    fop_setsecattr setsecattr
14941     fsinfo           genunix                       fop_dispose dispose
14942     fsinfo           genunix                       fop_dumpctl dumpctl
14943     fsinfo           genunix                        fop_pageio pageio
14944     fsinfo           genunix                      fop_pathconf pathconf
14945     fsinfo           genunix                          fop_dump dump
14946     fsinfo           genunix                          fop_poll poll
14947     fsinfo           genunix                        fop_delmap delmap
14948     fsinfo           genunix                        fop_addmap addmap
14949     fsinfo           genunix                           fop_map map
14950     fsinfo           genunix                       fop_putpage putpage
14951     fsinfo           genunix                       fop_getpage getpage
14952     fsinfo           genunix                        fop_realvp realvp
14953     fsinfo           genunix                         fop_space space
14954     fsinfo           genunix                        fop_frlock frlock
14955     fsinfo           genunix                           fop_cmp cmp
14956     fsinfo           genunix                          fop_seek seek
14957     fsinfo           genunix                      fop_rwunlock rwunlock
14958     fsinfo           genunix                        fop_rwlock rwlock
14959     fsinfo           genunix                           fop_fid fid
14960     fsinfo           genunix                      fop_inactive inactive
14961     fsinfo           genunix                         fop_fsync fsync
14962     fsinfo           genunix                      fop_readlink readlink
14963     fsinfo           genunix                       fop_symlink symlink
14964     fsinfo           genunix                       fop_readdir readdir
14965     fsinfo           genunix                         fop_rmdir rmdir
14966     fsinfo           genunix                         fop_mkdir mkdir
14967     fsinfo           genunix                        fop_rename rename
14968     fsinfo           genunix                          fop_link link
14969     fsinfo           genunix                        fop_remove remove
14970     fsinfo           genunix                        fop_create create
14971     fsinfo           genunix                        fop_lookup lookup
14972     fsinfo           genunix                        fop_access access
14973     fsinfo           genunix                       fop_setattr setattr
14974     fsinfo           genunix                       fop_getattr getattr
14975     fsinfo           genunix                         fop_setfl setfl
14976     fsinfo           genunix                         fop_ioctl ioctl
14977     fsinfo           genunix                         fop_write write
14978     fsinfo           genunix                          fop_read read
14979     fsinfo           genunix                         fop_close close
14980     fsinfo           genunix                          fop_open open

The arguments to the probes are:
• args[0]: fileinfo_t *
• args[1]: Return value (0 for success, in the case of writes this is the write size in bytes (ssize_t)).

So lets look at the most generic script we could write:

#pragma D option quiet

fsinfo:genunix:: 
{ 
printf("%s (%s) %s: %d [%s]n", probename, execname, args[0]->fi_pathname, args[1], args[0]->fi_fs);
}

The output specifies the probename (operation), process name that generated the op, the pathname if applicable, the return value, and finally the filesystem type:

$ dtrace -s fsinfo.d 
lookup (idmapd) /etc: 0 [ufs]
lookup (idmapd) /etc/resolv.conf: 0 [ufs]
getattr (idmapd) /etc/resolv.conf: 0 [ufs]
poll (idmapd) <unknown>: 0 [sockfs]
poll (idmapd) <unknown>: 0 [sockfs]
poll (idmapd) <unknown>: 0 [sockfs]
...
close (mysqld) /local/zone/root/var/tmp/#sql_47d3_0.MYI: 0 [zfs]
close (mysqld) /local/zone/root/var/tmp/#sql_47d3_0.MYD: 0 [zfs]
lookup (mysqld) /local/zone/root/var: 0 [zfs]
lookup (mysqld) /local/zone/root/var/tmp: 0 [zfs]
...

This can quickly be jazzed up by adding a conditional by zonename or filesystem type or process. We could do some aggregations rather than the play-by-play.

While the fsstat command can give you per-operation counts by mount, this gives you a lot more data which in turn can be used to find unusual I/O's that you didn't expect or help you construct realistic benchmarks using tools like FileBench.

> Read More... | Digg This!

Last nite I went to the OpenSolaris Night Seminar in Tokyo, which is conveniently held just a few floors above my office. Project Crossbow was the main topic of conversation from engineers Mami Sueki and Junko Yoshida, who also received certificates as new OpenSolaris Evangelists (yes, you have to earn your way around here) from Akira Ohsone and Shoji Haraguchi. The talks were streamed live and Shoji recorded everything so check his YouTube page in a few days for the video.

> Read More... | Digg This!