Org:Institute/SystemsPeach

Problems!

If the graphical display dosnt start - Your kernel maybe updated but not the nvidia drivers.

...press Ctrl+Alt+F1
...Login under your own username

 sudo apt-get install linux-restricted-modules
 sudo /etc/init.d/gdm restart

If you cant log in to your gnome desktop - bug https://bugs.launchpad.net/ubuntu/+bug/164804

...press Ctrl+Alt+F1
...Login under your own username 

 sudo rm /tmp/*
 sudo /etc/init.d/gdm restart

Server

The server is a simple install of ubuntu gutsy 64bit server addition

hostname: biserver

username: user

password <secret>

User Setup

All users are added on the server, as normal unprivilidged users, accessed via NIS, The users will have sudo access on their own computers to be able to install software and edit configuration files.

These users will be added using the following commands on the server

Adding the groups

 sudo addgroup biusers
 sudo addgroup biadmins # not used yet

Adding the users

 sudo adduser --ingroup biusers $BIUSER

Where $BIUSER are...

andy
cessen
sago
env
william
margreet
ton
brecht
ideasman42

NFS Server

Use for sharing home files

 sudo apt-get install nfs-kernel-server nfs-common portmap
 sudo dpkg-reconfigure portmap
 sudo /etc/init.d/portmap restart

Add a home share so NIS can have the users home directories mounted remotely.

 # run the command 
 sudo nano /etc/exports
 # Add the line at the bottom of the file
 /home 192.168.1.0/255.255.255.0(async,no_subtree_check,rw,insecure)
 /media/data/shared 192.168.1.0/255.255.255.0(async,no_subtree_check,rw,insecure)

Samba Server

This is used for the shared directory - so windows users can also access it.

smb.conf

[global]
security = user
hosts allow = ALL
null passwords = yes

[shared]
comment = Documentation Samba Server
path = /media/data/shared
read only = No
writable = Yes
guest ok = Yes
guest only = Yes

after changing settings run...

 sudo /etc/init.d/samba restart

SVN Server

Following this guide.

 sudo apt-get install subversion
 mkdir /media/data/svnroot
 svnadmin create /media/data/svnroot/peach
 sudo useradd subversion
 chown -R root:subversion /media/data/svnroot
 chmod -R ug+rw /media/data/svnroot

Add /etc/init.d/svnserve with this contents.

 sudo chmod +x /etc/init.d/svnserve
 sudo /etc/init.d/svnserve start
 sudo update-rc.d svnserve defaults

Uncomment the "password-db = passwd" line in /media/data/svnroot/peach/conf/svnserve.conf.

Add some users in /media/data/svnroot/peach/conf/passwd.

Backup Script

#!/bin/sh

TARGET="/media/backup"
mount /dev/sdb1 $TARGET

TEST="0"
for i in `cat /proc/mounts | cut -d' ' -f2`; do
  if [ "$i" == "$TARGET" ]; then
    echo "found mount..."
    TEST="1"
  fi
done

if [ "$TEST" == "0" ]; then
  echo "drive not mounted"
  exit 1
fi

cd $TARGET

mkdir home
mkdir system
mkdir data

cd ~

umount $TARGET


rsync -avx --delete --exclude "*_nobackup/" /                   /media/backup/system
rsync -avx --delete --exclude "*_nobackup/" /home/              /media/backup/home
rsync -avx --delete --exclude "*_nobackup/" /media/data/        /media/backup/data

Build Blender

This script (build.sh) can be configured to run as a cron job.

PREFIX="/media/data/shared/blender_nobackup"
cd $PREFIX/blender
svn update
svn revert -R .

REV=$(svnversion)
if [ -f $PREFIX/install/linux2/blender_$REV ]; then
	echo "Nothing to build"
else
	LOG_DEBUG=$PREFIX/build_log_debug_$REV.txt
	LOG=$PREFIX/build_log_$REV.txt


	### Remove all unneeded files??

	## IFS="
	## "
	## for f in $(svn stat | grep '?\t*')
	## do
	## 	f=$(echo $f | cut -c8-) # remove the ? and tab
	## 	echo "Removing unknown file "$f
	## 	rm -rf $f
	## done

	# Debug
	## nice -n19 scons clean BF_DEBUG=1 
	nice -n19 scons BF_FANCY=0 WITH_BF_YAFRAY=0 WITH_BF_GAMEENGINE=0 WITH_BF_FFMPEG=1 BF_DEBUG=1 \
		BF_BUILDDIR="../build_debug/linux2" \
		BF_INSTALLDIR="../install_debug/linux2"    1> $LOG_DEBUG 2>&1

	mv $PREFIX/install_debug/linux2/blender $PREFIX/blender/blender_debug_$REV
	rm blender_debug_current 2> /dev/null
	ln -s blender_debug_$REV blender_debug_current

	# Optimized
	## nice -n19 scons clean
        nice -n19 scons BF_FANCY=0 WITH_BF_YAFRAY=0 WITH_BF_GAMEENGINE=0 WITH_BF_FFMPEG=1    1> $LOG 2>&1

        mv $PREFIX/install/linux2/blender $PREFIX/blender/blender_$REV
	strip -s $PREFIX/blender/blender_$REV # make it a bit smaller
        rm blender_current 2> /dev/null
        ln -s blender_$REV blender_current

fi

Client

For the client we used Ubuntu gutsy 64bit edition.

The client can be created an mirrored across to many others. as sell as network authentication.

The following steps are required to setup the client systems.

Bios

• the computers have USB Keyboard disabled by default, this needs to be enabled so the USB keyboards that came with the computer work.
• The CDROM needs to be set as the first boot device.

OS (Ubuntu)

• when the boot CD shows its menu, press F6 and remove the argument "splash" and press enter.
• OS Ubuntu Gutsy (mostly a default install), US keyboard + language

Workaround for "end of partition error"

• • dont partition the drive from ubuntu, when the asking for guided or manual partition, open a terminal instead (leave the installer running)
  • type "sudo fdisk /dev/sda"
  • remove all existing partitions
  • add an extended partition the size of the disk (use default values)
  • add the following logical partitions...
    [+8000M : mount /] (os)
    [+8000M] (spare for dual boot)
    [+4000M] (swap)
    [+300000M : mount /media/data] (remaining space, work files)
  • set the type of partition 7 (the 2 gig partition) to swap type (82)
  • save the changes and close the terminal.
  • Now select manual in the ubuntu installer.
  • Make sure all ext3 partitions have the formatted option enabled.
• set the username and password to both be "guest"
• The OS should install now.
• Once installed youll be prompted to restart your system

Workaround for blank screen on load (disable splash)

• hold Esc key (as the system boots) to open the grub menu, and select recovery mode
• once logged in edit "sudo nano /etc/boot/grub/menu.lst"
• remove the argument "splash" from the kernel /boot/..... lines (at the bottom of the file) and from the commented line "# kopts....." so the splash will not show. removing from the commented line means the splash will not be added back after running update-grub.
• save, exit then reboot.

Remove CDROM from the sources.list

use the interface to do this, System, Administration, Software Sources

untick the CD as a source.

NVidia drivers

Install the restricted drivers AFTER upgrading all packages.

The quadro fx 1700 cards would flicker and run slow sometimes,
to fix this remove the nvidia drivers...

 sudo apt-get purge linux-restricted-modules
 sudo apt-get purge nvidia-glx-new

Installing the beta drivers from nvidia's site.

NIS Server

install portmap and nis

 sudo apt-get install portmap nis

hosts

add this line to /etc/hosts

192.168.1.76 biserver

The server does not have a static IP so if it ever changes, update the IP in this file.

hosts.allow

add this line to /etc/hosts.allow

portmap : ALL

This means that the computer will accept a NIS connection from the server, if the server had a static IP it would be better to use that.

Set up name services to use NIS

Edit /etc/passwd to add a line at the end saying:

+::::::

Edit /etc/group to add a line at the end saying:

+:::

Edit /etc/shadow to add a line at the end saying:

+::::::::

yp.conf

Edit /etc/yp.conf and add the line:

ypserver biserver

Restart NIS

/etc/init.d/nis restart

Changing Users/Passwords

 sudo make -C /var/yp

NFS Client

This is used to mount the remote home directory. (coupled with NIS)

 sudo apt-get install portmap nfs-common

fstab

Add this to the fstab

biserver:/home /home nfs rsize=8192,wsize=8192
biserver:/media/data/shared /shared nfs rsize=8192,wsize=8192

Explanation for Shared

• biserver - is the server hostname
• /media/data/shared - is the share on the server (use the servers home dir)
• /shared - is also where we mount the filesystem
• nfs - is the filesystem type
• rsize=8192,wsize=8192 - recommended settings for good performance.

Now you may need to restart services using the following command

 sudo /etc/init.d/portmap restart
 sudo /etc/init.d/nfs-common restart

SVN

Installation

 sudo apt-get install subversion rapidsvn

GUI

open RapidSVN in the Applications > Programming menu
open Repository menu, choose Checkout ...
URL: svn://biserver/peach (or svn://biserver/apricot)
Destination Directory: /media/data/peach (or /media/data/apricot)
click OK, enter username and password

Command Line

 cd /media/data
 svn co --username <username> svn://biserver/peach (or svn://biserver/apricot)
 enter password
 cd peach

Samba

Samba is used for shared directory for everyone to access.

Most of samba is configured on the server however an entry to the fstab is needed for automounting the shared directory.

install samba fs support

 sudo apt-get install smbfs

mount samba (removed! hangs gnome after sleep)

This will mount the samba share but it would not work after waking up so its better to use NFS fstab.

//biserver/shared /shared smbfs rw,guest 0 0

User Setup

User access to audio

By default the new users cant access the system autio to play music. here are the steps to solve this.

add this line to /etc/pam.d/common-account

auth optional pam_group.so

add this line to /etc/security/group.conf

*;*;*;Al0000-2400; audio,floppy,video,cdrom,plugdev,irc,scanner,src

Sudo Users

As unprivileged users, installing applications and configuring the system is not possible. however users may have privileges to configure their own workstation.

do this by adding the group "biusers" to the allowed sudo group.

run this on the client...

 sudo visudo

add this line to the bottom of the file.

%biusers ALL = ALL

Default permissions

Once the systems are running, edit ~/.profile and uncomment the line

# umask 022

and set to

umask 000

Also add the file ~/.gnomerc and add the line above to it. this means gnome apps will write files everyone can read on shared dirs.

Blender Compile

These commands can be copied into a terminal to compile blender on ubuntu gutsy.

sudo apt-get install scons subversion openexr libopenexr-dev build-essential libjpeg-dev \
libpng12-dev libopenal-dev libalut-dev libsdl-dev libfreetype6-dev libtiff-dev python-dev \
gettext libxi-dev; \
svn checkout https://svn.blender.org/svnroot/bf-blender/trunk/blender blender; \
cd blender; \
echo "BF_OPENAL_LIB = 'openal alut'" > user-config.py; \
scons -j4;

Server Build Script

PREFIX="/media/data/shared/blender_nobackup"
cd $PREFIX/blender
svn update
svn revert -R .

REV=$(svnversion)
if [ -f $PREFIX/install/linux2/blender_$REV ]; then
	echo "Nothing to build"
else
	LOG=$PREFIX/build_log_$REV.txt

	IFS="
	"
	for f in $(svn stat | grep '?\t*')
	do
		f=$(echo $f | cut -c8-) # remove the ? and tab
		echo "Removing unknown file "$f
		rm -rf $f
	done

	nice -n19 scons BF_FANCY=0 BF_DEBUG=1 WITH_BF_YAFRAY=0 WITH_BF_GAMEENGINE=0 WITH_BF_FFMPEG=1 1> $LOG 2>&1

	mv $PREFIX/install/linux2/blender $PREFIX/install/linux2/blender_$REV
	rm $PREFIX/install/linux2/blender_current # remove the symlink
	ln -s blender_$REV $PREFIX/install/linux2/blender_current
fi

Crash Reporting

Automatically report crashes from applications in /shared/bin to /shared/bugs/crash_reports/. Client installation:

/shared/bugs/crash_reports/install_script/install.sh

The patch to /usr/share/apport/apport:

--- original_apport 2007-10-16 18:16:56.000000000 +0200
+++ new_apport  2007-10-16 19:53:36.000000000 +0200
@@ -14,7 +14,7 @@
 # the full text of the license.

 import sys, os, os.path, subprocess, time, traceback, tempfile, glob
-import signal, inspect, grp, fcntl
+import signal, inspect, grp, fcntl, time

 import apport, apport.fileutils

@@ -285,9 +285,11 @@
     else:
         error_log('executable: %s (command line "%s")' %
             (info['ExecutablePath'], info['ProcCmdline']))
+   
+    blendercrash = info['ExecutablePath'].startswith("/shared/bin")

     # ignore non-package binaries
-    if not apport.fileutils.likely_packaged(info['ExecutablePath']):
+    if not (apport.fileutils.likely_packaged(info['ExecutablePath']) or blendercrash):
         error_log('executable does not belong to a package, ignoring')
         # check if the user wants a core dump
         drop_privileges(pid)
@@ -312,9 +314,15 @@
     # Create crash report file descriptor. We prefer to create the report in
     # report_dir if we can create a file there; if not, we just use stderr.
     try:
-        report = '%s/%s.%i.crash' % (apport.fileutils.report_dir, info['ExecutablePath'].replace('/', '_'), pidstat.st_uid)
+        if blendercrash:
+            report_dir = "/shared/bugs/crash_reports" # "/media/data/crash"
+        else:
+            report_dir = apport.fileutils.report_dir
+
+        timestr = time.strftime("%a_%d_%b_%H:%M:%S", time.localtime())
+        report = '%s/%s.%i.%s.crash' % (report_dir, info['ExecutablePath'].replace('/', '_'), pidstat.st_uid, timestr)
         if os.path.exists(report):
-            if apport.fileutils.seen_report(report):
+            if blendercrash or apport.fileutils.seen_report(report):
                 # do not flood the logs and the user with repeated crashes
                 crash_counter = apport.fileutils.get_recent_crashes(open(report))
                 crash_counter += 1
@@ -330,8 +338,13 @@
                     write_user_coredump(pid, cwd, core_ulimit)
                 sys.exit(1)
         reportfile = open(report, 'w')
-        os.chmod(report, 0000)
-        os.chown(report, pidstat.st_uid, pidstat.st_gid)
+        if not blendercrash:
+            os.chmod(report, 0000)
+            os.chown(report, pidstat.st_uid, pidstat.st_gid)
+        else:
+            # dirstat = os.stat(report_dir)
+            # os.chown(report, dirstat[4], dirstat[5])
+           pass
     except (OSError, IOError):
         report = None
         reportfile = sys.stderr

Video Settings - Compressing DV to OGG Theora (for the blog)

When exporting blender movies use ffmpeg codec.

Format MPEG-2
Bitrate 10000
Min Bitrate 0
Max Bitrate 10000

Audio

Enable Multiplex 
Format PCM

...leave other settings as is

ffmpeg2theora converts from almost any video format (DV, MOV files etc) into ogg.

ffmpeg2theora can be installed

sudo apt-get install ffmpeg2theora

From the cheap and nasty video camera

ffmpeg2theora clip.dv --audiobitrate 32 --channels 1 --samplerate 22050 --videobitrate 600 --optimize -x 360 -y 288

For DV on Andys camera we used this...

ffmpeg2theora weeklyNov16.avi --audiobitrate 32 --channels 1 --samplerate 22050 --videobitrate 800 --optimize --deinterlace -x 720 -y 400

To upload to google, we need a small MPEG file...

mencoder weeklyNov16.avi -o weeklyNov16_small.avi -vop scale=720:400 -ovc lavc -oac lavc lavcopts=vcodec=mpeg4:vbitrate=900:abitrate=32

Letterbox so google video plays properly

mencoder weeklyNov16.avi -o weeklyNov16_small2.avi -vf scale=720:400,expand=720:576 round=2 -ovc lavc -oac lavc lavcopts=vcodec=mpeg4:vbitrate=900:abitrate=32

Video compression, (high quality content)

For the peach trailer and high quality files.

For trailers and high definition content First make an avi with lossless jpeg compression and no sound from rendered PNG's

mencoder mf://*.png -mf w=1920:h=1080:fps=25:type=png -ovc lavc -lavcopts vcodec=ljpeg -oac copy -o test.avi

Now mix the audio with the avi

mencoder -oac copy -ovc copy -o test.avi -audiofile test.wav output.avi 

From this we can make a high and low quality ogg

ffmpeg2theora output.avi --optimize -v 8 -o output_big.ogg
ffmpeg2theora output.avi -x 720 -y 400 --optimize -v 8 -o output_small.ogg

For an avi that can play on windows xp without downloading any codecs

mencoder test.avi -oac lavc -ovc lavc -lavcopts vcodec=msmpeg4:vbitrate=6000:vhq:acodec=ac3 -o test-win-compat.avi

Making a DVD

This is a test to make the dvd from the trailer

Followed the instructions from http://www.linux.com/articles/53702

Unessential's

add the repositry - ref http://phorolinux.com/how-to-install-non-free-multimedia-codecs-on-ubuntu-710-gutsy-gibbon.html

Codecs

sudo apt-get install \
gstreamer0.10-alsa \
gstreamer0.10-esd \
gstreamer0.10-ffmpeg \
gstreamer0.10-gnomevfs \
gstreamer0.10-pitfdll \
gstreamer0.10-plugins-bad \
gstreamer0.10-plugins-bad-multiverse \
gstreamer0.10-plugins-base \
gstreamer0.10-plugins-base-apps \
gstreamer0.10-plugins-good \
gstreamer0.10-plugins-ugly \
gstreamer0.10-plugins-ugly-multiverse \
gstreamer0.10-tools \
gstreamer0.10-x \
libgstreamer-plugins-base0.10-0 \
libgstreamer0.10-0 \
totem-gstreamer

Flash

sudo apt-get install flashplugin-nonfree

MS Fonts

sudo apt-get install msttcorefonts

Cloning the client disk

First remove all cached packages so the disk image is smaller.

sudo apt-get clean

Edit the /etc/fstab and replace all device unique ID's (UUID's) since they will break the system when copying. Also replace UUID's in /boot/grub/menu.lst with the boot device (/dev/sda5 in our case)

reboot with a ubuntu install CD - you cant backup on a live system because of udev, proc etc.

cd /media/sda5 # where this is the OS partition
sudo tar cvpjf /media/storage/linux2.tar.bz2 ./ # where /media/storage is a spare filesystem, in our case /dev/sda8

Now this can be copied to the shared samba drive

Use the ubuntu install CD's to boot up another PC and partition it exactly the same way as the imaged PC's. now copy the archive onto the large partition of the fresh systems local drive (the 300gig partition) and then extract the files onto the first partition where the OS is installed.

 # format and mount the OS partition, and large free space and then extract the archive.
 sudo mkfs.ext3 /dev/sda5 # small os partition
 sudo mkfs.ext3 /dev/sda8 # large free space
 # ... mount the drives, youll need to create directories in /media for this.
 cd /media/sda5
 sudo tar xvpfj linux2.tar.bz2