Ok, so if you are currently using an A1 Pandaboard and the Ubuntu kernel, you will see the kernel has only allocated ~768Mb of the actual 1Gb of ram, this is due to a segfault when trying to compile things using 1Gb of ram.
SD Card speed issue
It seems either Ubuntu or ARM kernels have an issue with the SD card controller on the Pandaboard.
How to speed up Ubuntu on the Pandaboard
Easiest step is install Xubuntu by entering this on the commandline
sudo apt-get install xubuntu-desktop
It will take a good long while to complete but then you will have this:
As well you can try disabling the swapfile by editing it out of the /etc/fstab file and rebooting.