bugSavane - Bugs: bug #4834, the frontend allow data characters...

 
 
Show feedback again

You are not allowed to post comments on this tracker with your current authentification level.

bug #4834: the frontend allow data characters that the backend cannot converted into proper xml data

Submitted by:  Mathieu Roy <yeupou>
Submitted on:  Mon 05 Dec 2005 11:18:41 AM UTC  
 
Category: TransversalStatus: Postponed
Severity: 4 - ImportantPriority: B - Low
Assigned to: Mathieu Roy <yeupou>Open/Closed: Open
Release: 1.3 prePlanned Release: 
Reproducibility: NonePrivacy: Public

Mon 05 Dec 2005 03:24:45 PM UTC, comment #4:

I was about to go with such test but it does not work: decode_utf8 behave differently depending on perl version. And the wanted behavior is available in older ones...
(see http://aspn.activestate.com/ASPN/Mail/Message/perl5-porters/2922678)

sub IsUTF8 {
return 1 if $_[0] eq 0;
return 1 if $_[0] eq '';
return 1 if decode_utf8($_[0]);
return 0;
}

Also, putting the content into comments does not work either: xml parsers like xsltproc does not want non-utf8 into comment.

I dont think there is any interesting solution for now.

http://search.cpan.org/~markf/Test-utf8-0.02/lib/Test/utf8.pm would do the trick nicely, but it is not about to be available for production servers.

So people with invalid data are likely to have to edit it...

Mathieu Roy <yeupou>
Project AdministratorIn charge of this item.
Mon 05 Dec 2005 02:41:29 PM UTC, comment #3:

That's the cause of the trouble. The solution is to put in comments non-valid data.

If people have browsers that send non properly encoded data, we cannot really help it and it would surely not be costless for us to guess which encoding they are truly using. .

Mathieu Roy <yeupou>
Project AdministratorIn charge of this item.
Mon 05 Dec 2005 01:52:31 PM UTC, comment #2:

Another possibility: after doing some test, it turns out that we dont enforce UTF8 encoding.

Which means someone can configure his problem to sent things that are not encoded in UTF8, while they should. And that's indeed a possible cause.

Mathieu Roy <yeupou>
Project AdministratorIn charge of this item.
Mon 05 Dec 2005 11:25:37 AM UTC, comment #1:

Somehow, it means that people are still able to insert non-UTF8 data in the database, or am I dreaming?

http://www.w3.org/TR/REC-xml/#charsets

Mathieu Roy <yeupou>
Project AdministratorIn charge of this item.
Mon 05 Dec 2005 11:18:41 AM UTC, original submission:

The frontend allows users to insert data that the backend, through XML::Writer, cannot convert.

The database content of is an example
https://savannah.cern.ch/bugs/?func=detailitem&item_id=13860
To really be able to reproduce the problem, one need a sql export as the one attached

The current solution is to deactivate the checks made by sv_export, so it actually accept to make the export even knowing that users will have to update their xml files before giving it an xml parser.

The best solution would be to check for such content, convert it to a neutral thing, add a comment around the impacted data mentioning the issue. On all cases found, such content was here because of copy/paste but was not inserted intentionnaly.

The current workaround to skip the checks is acceptable for the release, but this last solution is the goal in the long run.
(And it would not be ok to imagine that the frontend could refuse characters by himself - when people do a copy an paste, we should in the item provide what they filled in the form)

Mathieu Roy <yeupou>
Project AdministratorIn charge of this item.

 

Attached Files
file #496:  prob.sql added by yeupou (2kB - text/plain - example)

 

Depends on the following items: None found

Digest:
   task dependencies.

 

Carbon-Copy List
  • -unavailable- added by yeupou
  • -unavailable- added by yeupou (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 12 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 15 Feb 2006 05:37:13 PM UTCyeupouPriorityC - Normal=>B - Low
      Severity3 - Normal=>4 - Important
    Wed 15 Feb 2006 05:36:55 PM UTCyeupouStatusConfirmed=>Postponed
    Mon 05 Dec 2005 03:24:49 PM UTCyeupouPlanned Release1.3=>
    Mon 05 Dec 2005 02:41:29 PM UTCyeupouStatusNeed Info=>Confirmed
      Assigned toNone=>yeupou
      Release1.3=>1.3 pre
      Planned Release=>1.3
      Open/Closed-Automatic update due to transitions settings-=>Open
    Mon 05 Dec 2005 11:20:08 AM UTCyeupouSummarythe frontend allow data characters that the backend cannot convert into proper xml data=>the frontend allow data characters that the backend cannot converted into proper xml data
    Mon 05 Dec 2005 11:18:41 AM UTCyeupouAttached File-=>Added prob.sql, #496
      Carbon-Copy-=>Added ype
    Show feedback again

    Back to the top


    Powered by Savane 3.1-cleanup