patchFreeciv - Patches: patch #3027, libicu

Show feedback again

patch #3027: libicu

Submitted by:  Marko Lindqvist <cazfi>
Submitted on:  Sun Nov 6 23:49:52 2011  
Category: generalPriority: 5 - Normal
Status: DonePrivacy: Public
Assigned to: Marko Lindqvist <cazfi>Open/Closed: Closed
Planned Release: 3.0.0Contains string changes: None

Add a New Comment (Rich MarkupRich Markup):

You are not logged in

Please log in, so followups can be emailed to you.


Thu Apr 16 23:56:48 2015, comment #5:

I got the linking problems in windows (crosser) setup sorted, so this could go forward in TRUNK.

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Sun Apr 7 23:04:26 2013, comment #4:

> In some ways it would be a lot easier if the internal character
> set could be relied on to be UTF-8.

Yes, now that I'm looking libicu stuff, I'd like to replace a lot of our most low-level string handling to use its UTF-8 handling. For one, I think we have at least theoretical bugs in how we currently handle strings that are actually of some-encoding as ASCII. Fixing those to work on with any internal encoding is not going to happen, but if we could rely to that everything is UTF-8 except in the very last moment it's written out to non-UTF-8 "device" or very first moment it's read in from, handling would be in most cases trivially simple (call equivalent icu function instead of c-lib function)

At present day, do we have any reason not to switch to UTF-8 that way in 2.6?

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Sat Mar 30 21:19:03 2013, comment #3:

Not only UTF-8, but stuff like collations.

As for portability concerns, I added icu build to crosser, and it at least builds fine for MinGW.

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Sat Jun 30 14:48:43 2012, comment #2:

(When I say "UTF-8" I might actually mean "Unicode".)

Jacob Nevins <jtn>
Project Administrator
Fri Jun 29 09:24:59 2012, comment #1:

In some ways it would be a lot easier if the internal character set could be relied on to be UTF-8. For instance I know of portable code to fix bug #17289 without introducing platform dependencies, but only assuming UTF-8. Right now I think I have to code as if it could be any character set (I might end up with Shift-JIS or something), and so be very conservative in my assumptions -- is that right? (I can't immediately locate the developer documentation on character sets, I'm sure I'm seen some somewhere.)

Jacob Nevins <jtn>
Project Administrator
Sun Nov 6 23:49:52 2011, original submission:

Should we take libicu to use for all kind of UTF-8 manipulation?

This came up after discussion about translations. Sometimes it's really hard to arrange sentences in translation so that it doesn't begin with some "%s", which uses word starting with lower case letter. It was requested that we would add some mechanism to upper case such first letters. Well, that's not trivial to do for variable-byte UTF-8 texts we have. Libicu would probably help a lot here.
We've also had, and still have, a lot of problems of truncating UTF-8 strings potentially mid-multibyte-character. On a somewhat related note, I'd like to change our naming of variables/constants/macros/whatever so that "len" means length of text in characters, and "size" is used when number of bytes are in question.

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.


(Note: upload size limit is set to 1024 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


   patch dependencies.

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by jtn (Posted a comment)
  • -unavailable- added by cazfi (Submitted the item)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Error: not logged in



    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu Jan 21 06:53:26 2016cazfiStatusNone=>Done
      Assigned toNone=>cazfi
    Thu Apr 30 00:55:12 2015cazfiDependencies-=>Depends on patch #3930
    Thu Apr 16 23:56:48 2015cazfiPlanned Release=>3.0.0
    Sun Jan 4 04:08:52 2015cazfiPlanned Release2.6.0=>
    Sun Apr 7 22:53:37 2013cazfiDependencies-=>Depends on patch #3838
    Sat Mar 30 21:19:03 2013cazfiPlanned Release2.5.0=>2.6.0
    Show feedback again

    Back to the top

    Powered by Savane 3.1-cleanup