OK, I spent some time getting a dual-head system going on a sun box I have at work. It's nothing fancy, but it makes development a little nicer. The trick was in knowing what needed to be done to get things set up right. So here are the steps so that I don't have to figure it out again.
I installed a second XVR-100 video card in the box and did a simple boot -r to get Solaris to see the device. Then I needed to configure the two devices so they were the same. The devices ended up being /dev/fbs/pfb0 and /dev/fbs/pfb1.
fbconfig -dev pfb0 -res 1280x1024x60
fbconfig -dev pfb1 -res 1280x1024x60
fbconfig -dev pfb0 -depth 24
fbconfig -dev pfb1 -depth 24
fbconfig -dev pfb0 -fake8 enable
fbconfig -dev pfb1 -fake8 enable
and you can check the configuration with:
fbconfig -dev pfb0 -propt
Now you need to change the Xservers file to have both devices and xinerama mode. To do this you need to make sure that /usr/dt/config/Xservers points to /etc/dt/config/Xservers and that there's a like that looks a lot like this in it:
:0 Local local_uid@console root /usr/openwin/bin/Xsun +xinerama
-dev /dev/fbs/pfb0 -dev /dev/fbs/pfb1 left -nobanner
...all on one line, of course. Now reboot. The CDE login screen should come up on one display but the mouse should move seamlessly between the two.
Now fix WindowMaker - the trick is knowing that xinit can allow you to completly specify the X server to use. So I made an alias to start WindowMaker like this:
alias wm 'xinit ~/.xinitrc-wmaker -- /usr/openwin/bin/Xsun
+xinerama -dev /dev/fbs/pfb0 -dev /dev/fbs/pfb1 left >>&! /dev/null'
with this, I can start it easily and get both heads working as they should.