If you run your own Gitlab server sooner or later you will be confronted with the message '500 We're sorry, but something went wrong. We've been notified about this issue and we'll take a look at it shortly.'
In my set of circumstances I got this screen immediately after a 'successful' login - i.e. I logged into Gitlab with credentials I irrefutably knew were correct. Gitlab is a beast with many different components and interactions and therefore tracking down, diagnosing, and fixing the error could have been a formidable task.
The best starting point is to have a look at the /home/git/gitlab/log/production.log file.
ActionView::Template::Error (Mysql2::Error: Can't create/write to file '/var/tmp/mysql.RCx5Ky/#sql_17c5_0.MYI' (Errcode: 2): SELECT `projects`.* FROM `projects` INNER JOIN `namespaces` ON `namespaces`.`id` = `projects`.`namespace_id` WHERE `projects`.`id` IN (1, 3, 4, 5, 6) ORDER BY projects.last_activity_at DESC LIMIT 20): 8: New project
openSUSE-123-64-minimal:/home/git/gitlab/log # df -k Filesystem 1K-blocks Used Available Use% Mounted on devtmpfs 248012 32 247980 1% /dev tmpfs 251672 0 251672 0% /dev/shm tmpfs 251672 26144 225528 11% /run /dev/sda3 19091164 2415148 15706240 14% / tmpfs 251672 0 251672 0% /sys/fs/cgroup tmpfs 251672 26144 225528 11% /var/run tmpfs 251672 26144 225528 11% /var/lock /dev/sda2 516040 44848 444980 10% /boot openSUSE-123-64-minimal:/home/git/gitlab/log #
No problems there. A little Googling and I located this thread http://serverfault.com/questions/69557/fixed-a-problem-on-a-rails-mysql-app-how-to-prevent-it-happening-again. The issue here is MySQL is closing the persistent connection opened by the Rails app. There are a number of timeout values in MySQL and as yet I am unsure which value I need to change here. I do know that I hadn't used my Gitlab server for a few weeks over the Xmas period but there had been occasions in the past when it had continued working with a few days inactivity.
The easiest workaround (for now) to get your site back up and running, is to simply reboot your MySQL server
# /etc/rc.d/mysql restart