The site was just down for about 3 minutes while I rebooted the database server. It might (or might not) run faster now.
The main storage server is down after being slow for most of the day. We are working on the situation and will have a status report within 30 minutes.
Update: Unfortunately we won’t know anything until the hosting provider’s day shift begins tomorrow morning.
We are trying an experiment with a new rotation mechanism that is a lot faster than the current image-level rotation, by using direct manipulation of the JPEG blocks. There seem to be some problems with it, particularly around the edges of photos, so those of you who are tired of slow rotations can try it out and see what you think.
Visit this link to enable the fast rotation for the rest of your session. Then rotate some photos! For most photos it should be noticeably faster, but there may also be some quality problems. Feel free to post comments or complaints here.
One note: For rotations that take the photo back to the original, we still do a traditional mogrify based rotate.
Since yesterday we have been giving newly uploaded photos a light sharpening. This only applies to the “standard” and “large” formats, i.e. the single-page photo format and the large format that is displayed when you click on the photo.
A lot of users have been asking for this sharpen, but please comment here if you don’t like this, or if you would rather have an unsharp masking instead.
Earlier today 23′s internet service provider lost its internet connection. The site has been down for approximately two hours.
The ISP’s own website is now back up, but 23 is not. We are working on the problem and will post updates here.
23 is back online. We apologize to all users for this very rare unscheduled downtime. We’ll be reviewing procedures together with our internet service provider to make it as unlikely as possible that this will happen again.
After our transition to the new servers in Amsterdam, the web server process seems to be dying every few minutes. We will try to resolve this issue, but if the service has not recovered within the next few hours we will transition back to the servers in Cologne.
Update: The service window has ended, but there are some stability problems right now. I’ll post an update here when they’re resolved.
The 23 service is undergoing a transition from our current servers in Cologne, Germany, to the new 23 data center in Amsterdam, the Netherlands. This is an announcement of a service window on Tuesday, December 6th, 2005.
For this reason 23 will be unavailable from 10:00 UTC.
The service window will last at least until 17:00 UTC. However, the service window may last longer. An extension of the service window will be announced here.
During the service window, 23 will be completely unavailable. However, photos uploaded by email should be queued for automatic processing after we are moved to Amsterdam.
In the event that the service window is postponed, an announcement will also be posted here.
Update: The service window has been postponed for one hour while we resolve a problem with the new database.
Update: The service window has been extended two hours.
Update: The service window has been extended another two hours. This will probably be the last extension.
Update: The service window has been extended another hour. Fingers crossed.
This post will contain information about our current migration to the new 23 data center in the Netherlands.
17:34 UTC Monday: We’re still in the process of copying data.
18:38 UTC Monday: I estimate that the copying process will take at least a couple more hours. If it’s done by midnight, I’ll announce downtime starting at 09:00 UTC on Tuesday.
18:46 UTC Monday: We’re currently pushing through 16 mbps between our Cologne and Amsterdam data centers. Earlier I measured 33 mbps. I love the Internet.
19:20 UTC Monday: You guys sure have a lot of photos! If we ever do this again, we’d probably have to move data the old-fashioned way.
22:00 UTC Monday: Copying won’t be done today. The copy job will continue through the night, and I’m announcing downtime starting at 09:00 UTC tomorrow.
22:13 UTC Monday: Here’s the service window annoucement.
06:02 UTC Tuesday: Copying is almost done. We’re on track to start the service window at 09:00 UTC.
08:45 UTC Tuesday: We’ve started the service window 15 minutes early to take advantage of the light traffic.
10:44 UTC Tuesday: I’ve started the final rsync run to ensure that the old and new systems have identical files.
13:14 UTC Tuesday: I had copied part of the data using scp without preserving permissions and modification dates, which meant that rsync was re-transferring those files. I’m now running my original copy job (with tar piped through ssh) on those files, around 18 GB, after which I’ll need to re-run rsync.
13:16 UTC Tuesday: People have asked in comments whether everything will be up, or whether it will take more time after the service window is ended. The answer is that everything should be back up by 16:00 UTC, but it’s always possible that I’ve made mistakes.
14:21 UTC Tuesday: The 18 GB I mentioned earlier have now been copied and I’ve started the rsync run.
15:09 UTC Tuesday: I just learned that chown doesn’t change ownership of symbolic links unless you pass the -h option, requiring a rsync restart and another hour. This is starting to get embarassing.
16:08 UTC Tuesday: The end is in sight. rsync is now transferring photos that were uploaded today.
16:15 UTC Tuesday: rsync is done. Now the rest depends on how fast I can type.
16:35 UTC Tuesday: The web server is set up again. I’m running a vacuum of the database. We should be up in a few minutes, apart from mail upload.
18:59 UTC Tuesday: The last problems with PostgreSQL 8.1 should be fixed now. The service is up again. It may be a bit slow for the next 12 hours because requests that hit the old IP address will be proxied to Amsterdam.
19:23 UTC Tuesday: The web server process is crashing every few minutes. Trying to work this out.
10:10 UTC Wednesday: The blog has been inaccessible for some people for at least 12 hours because I made a mistake in the DNS configuration. After the web server came up, the server process kept dying with a segmentation fault (“fatal signal 11″) every few minutes. I tried a number of things and will try to identify the precise cause of the error later, but it seems to be either an insufficient stack size or the version of tDOM that we are running. I changed the stack to 2 MB and upgraded to the CVS version of tDOM, and things seem to be running fine now. Zip downloads and url uploads will continue to be deactivated, probably for the rest of the week, until we get the second web server online.
14:26 UTC Thursday: We’ve had some serious performance issues since the transition. We isolated the cause to a maxthreads setting in AOLserver that was way to high. It’s been lowered to 35 (from 50) and things seem to be faster now. The second web server will probably be online this evening. This will improve performance while many users are uploading.
We’ve just added RSS 2.0 enclosures to our RSS feeds. This means that we have added a special link to a large version of each image to 23′s RSS feeds. Applications can read this special link and do interesting stuff with it — for example it may automatically add photos to iPhoto.
The popular podcasting client iPodderX can do this. In iPodderX’s preferences window, select the “Exporting” tab and check “Move Images to iPhoto After Downloading”. Then subscribe to your friends’ RSS feeds on 23. In the future their photos will then appear in iPhoto whenever you sync iPodderX.
Read more about “image podcasting” on Peter Forret’s site.
 | The Visual Sharing Company |
We create tools allowing individuals and organizations to unleash the power of visual sharing.
