Hum, I'm forced to do a 1.0.7+2 since 1.0.7+1 tag is already used in the cvs :(

Would be mysqlhotcopy helpful?

I'll try tomorrow to make an install of 1.0.6, then try the upgraded script with binaries in attachement, we'll see.

if it's ok, we'll repackage 1.0.7+ and 1.0.8+. At the same time, I would be nice to check if iconv can undo the change.

Hi Mathieu,

I think the fix for this bug (adding --hex-blob to the mysqldump program options) should be everything that is needed for updates from 1.0.6. As I said, it does not currently fix the databases which have already been converted.

I've added a new release version in the CVS, the tag is called "REL_1-0-7+1". Feel free to check this out and have a look yourself. I think you can package a new release based on that tag.

If we need to repackage 1.0.7 to avoid database breakage (and we do), it must be done ASAP.
As soon as a working solution to avoid this problem during conversion is found, the repackaging should be done.

And 1.0.8 must be also repackaged, since it includes update scripts.

I moved packages to directories called WARNING_NEED_FIX since we cannot affort to let people breaking their installation knowning it.

How to fix already broken things, that's secondary for now. I dont think there's any solution apart using backups and I very glad this bug get found now and not later. I wonder how come we forgot to check that :(

What would be nice would be to write a small script that could take a mysql dump of a whole database, take as argument a number of supposedly the more recent broken attachment, and would prepare a .sql file that would contain what is necessary to reinsert the appropriate content.
In Gna! case, fortunately there are very few attachment in the database; I dont think we have still backups of database of 5 month ago.

Indeed, the first thing to do is too check whether iconv can reverse the change.

Tobias, do you think you have time to check on those issues in a timely fashion? Since you are the one that is most familiar with all these UTF8 issues, that would be helpful.

User reported corrupted downloads at Savannah today.

Binary data was encoded utf-8 during the database conversion.

I reverted those using a backup I luckily kept since the conversion. This data might be restored using the reverse iconv transformation, but I haven't tested. Downloads could not be reverted by simple users, because Savane truncates the (bigger, if iso8859->utf8) data at file_size.

Apparently trackers attachments are the only binary data. There might be issues with binary-encoded numbers, but apparently we don't use that.

We have a problem with the Savane upgrades: I got 2 issues with 1.0.7 and another one with 1.0.8. In each case one need to be a Savane dev to fix the issue so that's pretty bad for us.

As for fixing the utf8 upgrade procedure, I guess we should look at MySQL tools to do the job, or else automate the attachment recovery.