bugFreeciv - Bugs: bug #19087, Create language statistics file

 
 
Show feedback again

bug #19087: Create language statistics file

Submitted by:  Jacob Nevins <jtn>
Submitted on:  Thu 01 Dec 2011 02:54:41 AM UTC  
 
Category: generalSeverity: 1 - Wish
Priority: 5 - NormalStatus: Fixed
Assigned to: Marko Lindqvist <cazfi>Open/Closed: Closed
Release: Operating System: Microsoft Windows
Planned Release: 2.6.0, 3.0.0

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 25 Mar 2015 06:06:28 PM UTC, comment #11:

Further problems noticed as bug #23395. Given that the benefit of this patch are rather small, and it seems to be very risky (even when known problems get fixed), better to leave it out from stable S2_5.

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Mon 16 Mar 2015 04:27:37 AM UTC, comment #10:

S2_5 build with bug #23382 fixes included.

(file #24083)

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Fri 13 Mar 2015 02:25:19 AM UTC, comment #9:

S2_5 version should be updated with bug #23382 part fixed before committing.

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Thu 12 Mar 2015 03:05:31 AM UTC, SVN revision 28501:

Generate langstat.txt as part of 'make dist'

See bug #19087

(Browse SVN revision 28501)

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Thu 12 Mar 2015 03:05:13 AM UTC, SVN revision 28500:

Generate langstat.txt as part of 'make dist'

See bug #19087

(Browse SVN revision 28500)

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Thu 12 Mar 2015 02:50:45 AM UTC, comment #6:

> Creates langstat.txt, but does not yet try to use it in Windows > Installer build process.


This ticket now about this langstat.txt creation step. Windows Installer part (the original issue of this ticket) created as patch #5931.

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Sun 08 Mar 2015 01:16:19 PM UTC, comment #5:

Patch written from scratch. Creates langstat.txt, but does not yet try to use it in Windows Installer build process.

Added Makefile of its own for bootstrap/. This means that 1) the directory gets created to builddir hierarchy too -> there's directory to build langstat.txt to 2) we can control its position in subdirectory listing while root Makefile was always the first one to get handled -> we create langstat.txt only after translation/ has been handled and po-files merged against current strings -> percentages reflect po-files in distribution, not the situation as in previously committed po-file version

Currently has only core translation domain statistics.

(file #24022, file #24023)

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Sun 22 Sep 2013 12:44:19 PM UTC, comment #4:

> And the main problem:
> - Works only with English locale when every .po file has at
> least 2 (=plural) of each of translated, fuzzy, and
> untranslated strings. This is due to very fragile method of
> parsing msgmsrg --stat output. Even if we fix it to work with
> current version of gettext, it would depend on gettext output
> strings never to change...

I'm not actually so worried about that.

This only needs to absolutely work on the system of the person who builds the distribution tarballs (me, currently).

I think if we force the locale for this command to C or en_US or something, and make the parsing script paranoid and bail out if it sees any text it doesn't completely understand how to parse, the risk of gettext's output changing under our feet is acceptably small as to be managed by the release manager doing a trial "make dist" run after major system upgrades / before committing to a new release.
(Of course if there's a better way to get stats in machine-readable format without awkward dependencies, I'm interested.)

This is going to want revisiting with the split of po-files in patch #4190. I think the desired outcome is that the installer includes stats for both "core" strings and extended nations (e.g., "Faroese (fo): core game translation 100% complete, extended nations 36% complete").

Jacob Nevins <jtn>
Project Administrator
Mon 30 Jul 2012 12:27:56 PM UTC, comment #3:

The most easiest part handled in patch #3463

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Sun 29 Jul 2012 11:48:51 AM UTC, comment #2:

Attaching new version just in case, though I think this approach will never be of acceptable quality.

- Added Indonesian to langnames.txt
- Added langnames.txt to tarball
- Stats generation in separate script instead of being incorporated to win32/Makefile
- langstat.txt now generated to $(top_builddir)
- langstat.txt generation PHONY target so gets always regenerated when "make dist" run

Bugs:
- langstat.txt is generated as first step of 'make dist', most importantly before .po files have been updated
- Requires bash (should work wiht any standard /bin/sh)

And the main problem:
- Works only with English locale when every .po file has at least 2 (=plural) of each of translated, fuzzy, and untranslated strings. This is due to very fragile method of parsing msgmsrg --stat output. Even if we fix it to work with current version of gettext, it would depend on gettext output strings never to change...

(file #16228)

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Wed 25 Jul 2012 02:11:31 AM UTC, comment #1:

Patch to produce langstat.txt file with language codes, translation percents, and language names. Code and name are taken from bootstrap/langnames.txt, translation percent calculated.

(file #16189)

Marko Lindqvist <cazfi>
Project AdministratorIn charge of this item.
Thu 01 Dec 2011 02:54:41 AM UTC, original submission:

Christian has made several improvements to language selection in the Windows installer recently, up to the point in 2.3.1 where a relatively nontechnical user can choose a language, provided they know their language code.

However, it could be slicker yet. Ideally I think the language selection process would include the following information for each language:

  • Filename / language code, as now
  • Language name, so people don't have to know language codes
    • In English at least, and possibly native too
    • Possible sources: Wikipedia ISO 639-1, Ubuntu /usr/share/language-selector/data/languagelist
  • Translation completeness stats
    • Currently one can't tell the difference between a 100% or 30% complete translation at install time, and I would expect that to influence the decision of whether to use it.

Realising all of these, particularly the last item, could benefit from some coordination between the standard tarball release process and the Windows-specific packaging process.

For instance, it would be natural for the standard release process to drop out some sort of file containing translation statistics right after building the po-files, for inclusion in the release tarball, in a form that the Windows packaging process could pick up, to minimise the need for gettext tools in the latter.

This approach means that addition of new languages can be done by any developer, and would also potentially allow other packaging processes (such as Mac OS?) to benefit from the same effort.

But I'm not sure of the details. I suspect having the latest Windows installer scripts or similar material under version control would help.

Jacob Nevins <jtn>
Project Administrator

 

(Note: upload size limit is set to 1024 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #24083:  Langstat-S2_5-2.patch added by cazfi (5kB - text/x-diff)
file #24022:  Langstat.patch added by cazfi (5kB - text/x-diff)
file #24023:  Langstat-S2_5.patch added by cazfi (5kB - text/x-diff)
file #16228:  Langstats-2.patch added by cazfi (3kB - text/x-patch)
file #16189:  Langstats.patch added by cazfi (2kB - text/x-patch)

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by cazfi (Updated the item)
  • -unavailable- added by akerbeltz
  • -unavailable- added by jtn (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 16 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 29 Mar 2015 10:58:59 PM UTCcazfiOpen/ClosedOpen=>Closed
    Wed 25 Mar 2015 06:06:28 PM UTCcazfiStatusReady For Test=>Fixed
      Assigned toNone=>cazfi
      Planned Release2.5.1, 2.6.0, 3.0.0=>2.6.0, 3.0.0
      SummarySlicker language selection in Windows installer=>Create language statistics file
    Mon 16 Mar 2015 04:27:37 AM UTCcazfiAttached File-=>Added Langstat-S2_5-2.patch, #24083
      StatusIn Progress=>Ready For Test
    Fri 13 Mar 2015 02:25:19 AM UTCcazfiStatusReady For Test=>In Progress
    Sun 08 Mar 2015 01:16:19 PM UTCcazfiAttached File-=>Added Langstat.patch, #24022
      Attached File-=>Added Langstat-S2_5.patch, #24023
      StatusNeed Info=>Ready For Test
      Planned Release=>2.5.1, 2.6.0, 3.0.0
    Sun 29 Jul 2012 11:48:51 AM UTCcazfiAttached File-=>Added Langstats-2.patch, #16228
    Wed 25 Jul 2012 02:11:31 AM UTCcazfiAttached File-=>Added Langstats.patch, #16189
    Thu 01 Dec 2011 09:52:15 AM UTCakerbeltzCarbon-Copy-=>Added akerbeltz
    Thu 01 Dec 2011 02:55:43 AM UTCjtnDependencies-=>Depends on bugs #19088
    Show feedback again

    Back to the top


    Powered by Savane 3.1-cleanup