Export and axportd - problem with filed multiline (Textarea)

[marcinw]

Hello @mbatz,
in my case, when there is a multiline field for a given type of object and it is filled (there is text separated by enter in keyboard (next line)) in objects export to csv, xlsx or exportd when the source is ExternalSystemCsv, every time i had wrong data in such files comes out because that enter in that field moving to csv or excel and no program can handle it.

i have 8 objects in this objects of type and i filled object similar like that:

and list of objects:

when i click export to csv or xslx i get this

i have 8 objects but in many entries

i should get csv, xlsx, ExternalSystemCsv for 8 objects in which i should saw only 8 entries and header so i should saw max 9 entries but there are many more.

You should think about how to fix enter in keyboard (next line) in filed multiline (textarea) when export to csv/xlsx.
The same situation is when i have created job in exportd and destination is ExternalSystemCsv.
Maybe You should think about change next line sign to example for 3-5 spaces or any different sign what You want?
I think You should fast fix that bug because that is important function and don’t work ok.

As long as you managed to deal with the new lines (enter) for the multiline (textarea) field in the list of items, as you can see on one of the screens in this topic, you forgot about the exports to csv and xlsx.

I only tested these functions, so I can talk about them. Perhaps errors with lines also appear in other cases. This has to be checked

[mbatz]

Hi @marcinw,

the problem here is, that the csv format has not a real standard and every tool does its own magic here. Some options are defined in RFC 4180. This RFC also mentions line breaks in fields and demands that fields with line breaks should be enclosed in double-quotes. That is what we are doing in our exports with textareas.

Some of the tools outside there can interpret these line breaks. In one of the last customer projects we did, we had some luck with LibreOffice Calc and also MacOS Numbers does a good interpretation of linebreaks in fields:

[marcinw]

I know what you mean, but you can set something up in your program when generating the csv export if multiline is used in the object so that there is no problem opening the csv with other programs like e.g. via Excel.
Very much people works with Excel and this software don’t want good interprate this csv file.

[marcinw]

If don’t want change that to work csv other programs only LibreOffice Calc maybe think about export to excel from exportd

[mbatz]

Hi @marcinw,

I’ll plan to do some work to check, if we can change some parameters in the CSV exports to also get a good support for Excel. But I don’t have high hopes on that, as it seems Excel behaves different for different localization settings.

Did the XLSX export on the object tables work for you? I think, we can also add a XLSX exportd class.

[marcinw]

Hello @mbatz
Unfortunately not
Export to xslx has bugs also.
in this case it is a separate line each time, but the errors are in the columns.

example:

In this example object with id 644 and 600 start-date move to column wybor, end-date to start-date , from column second-name to text-area-field and in column second name is None.

when i export from bigger list of objects and other types there are many more errors of this kind and in different columns

if you need to debug that case with moving value to other columns in export to xlsx i can export this type and export objects for that type

[mbatz]

Hi @marcinw,

today I had a deeper look at the file exporter and fixed some issues there (NET-732 and NET-733). There were some issues in the different exporters (XLSX, CSV but also JSON and XML), if fields were removed or added to an object type definition with existing objects. These issues caused the behavior with having values in wrong columns in the exports. The bugfixes are currently in our testing/code review phase and will make it into our next bugfix release 1.4.3, which I’ll plan for the next days.

[marcinw]

Hi @mbatz,
Ok so i will wait and check after update.
I hope what you wrote will be good. I know this is a small excerpt from the entire assignment that I submitted, but in small steps to the end. I hope that what is the main goal will be achieved as well.
It is worth correcting the multiline (change enter to spaces (2-4), or another fixed character for csv (excel, notepad ++ cannot interpret like LibreOffice you wrote about),
or correct export to excel completely and add exportd to excel functionality

Best Reagards
MArcin

[mbatz]

Hi @marcinw,

version 1.4.3 is published. That will fix the issue with the XLSX file exporter. I also added a user story to add a new exportd class which will create XLSX files (NET-743). This should not be a big effort and maybe, we can add that to the 1.5 release.

I don’t see a real chance to get improvements for the CSV exports. The only things we can change here is the delimiter (use a comma or semicolon or anything else) or the quotation of fields. As there is no real standard, it is hard to find a way to support a bunch of tools.

[marcinw]

Hi @mbatz,
i will check this version and will get you know.
Export is in your program from the begining and it is deafult with coma. Anyone from this place can’t change which fields wants to export to some format.
This means that if anyone uses the textarea fields and also uses them to describe in created objects, then when they generate a report using the basic function export to csv
every time it will have problems with interpretation in programs (except LibreOffice and Mac) and data will be bad.
There are a lot of people using MS Office, it seems that you have to take this into account.

The comma and semicolon are usually the most frequently used separators to separate fields in such formats as exports and exportd to csv files, so this cannot be done like this.
Sorry i can’t add files so add pictures
export to csv

exportd to csv

The best thing to do if you don’t want to interfere too much as long as you try to fix it is either use one space or a blank character.
You have already done so in the DocAPI functionality. When you want to make a pdf about an object, it deletes new lines and puts them on one line replacing the newline with a space.

and object look that

It’s a bit weird that it works like this in one place and not in the other.

And to make it more interesting, in DocAPI it should keep newlines (which I wrote about before on other topic), and in exports to csv it should not be such a situation because here each new line is a separate object, so it’s the other way around

[marcinw]

Hello,
Didn’t this module developed for version 1.5.0?

@mbatz

Hi @marcinw,

version 1.4.3 is published. That will fix the issue with the XLSX file exporter. I also added a user story to add a new exportd class which will create XLSX files (NET-743). This should not be a big effort and maybe, we can add that to the 1.5 release.

I don’t see a real chance to get improvements for the CSV exports. The only things we can change here is the delimiter (use a comma or semicolon or anything else) or the quotation of fields. As there is no real standard, it is hard to find a way to support a bunch of tools.

[mheumueller]

Hi @marcinw,
Unfortunately, we had no resources left for NET-743. So this will probably be implemented in one of the minor versions 1.5.2 or 1.5.3.