WCPS responses do not add EOL at the end of the response

[Piero Campalani]

newline (\n, 0x0a) should be put at the end of a CSV encoded WCPS response.

By default, vim automatically adds it on save, so systemtests can become unreliable when oracles are manually edited.

This currently inhibits multipoint WCPS tests to pass.

$ echo `pwd`
/home/rasdaman/rasdaman/systemtest/testcases_services/test_wcps
$ hexdump -C output/62-subset-all-dims-multipoint.test.out | tail -2
00000800  2c 32 35 35 2c 32 35 35  7d                       |,255,255}|
00000809
$ hexdump -C oracle/62-subset-all-dims-multipoint.test.oracle | tail -2
00000800  2c 32 35 35 2c 32 35 35  7d 0a                    |,255,255}.|
0000080a
$ hexdump -C oracle/01-just_trim.test.oracle | tail -2
00000130  49 45 4e 44 ae 42 60 82                           |IEND.B`.|
00000138

[Piero Campalani]

• Solution 1: CSVs with EOL at the end of the response
  • fix Petascope
• Solution 2: CSVs without EOL at the end of the response
  • fix multipoint-related templates (remove EOL)

[Dimitar Misev]

No WCPS should not add EOL in my opinion, it's better to fix the tests to match the output.

[Piero Campalani]

I don't have particular preferences, I just thought it was good practice for text files and ​internet protocols.

Why would you keep no-eol?

[Dimitar Misev]

I think you'd want to end the line with a new line when the next line holds more information, and is not just an empty line that is the end of output?

Peter do you maybe have any opinion on this, should scalar output (or csv encoding) from WCPS end with a new line?

[Dimitar Misev]

We could modify the diff command in the systemtest as well to ignore whitespace, I think we don't have any output that is critical of whitespace (like Python code for example).

[Dimitar Misev]

It's an issue of semantics, e.g. from ​wikipedia:

Two ways to view newlines, both of which have valid internal logic, are that newlines terminate lines or that they separate lines. If a newline is considered a separator, there will be no newline after the last line of a file.

We don't have multiline output from WCPS so in that sense it would be valid for the output to end with a \n

[Peter Baumann]

I understand that this affects only CSV? Its definition in RFC 4180 (​http://tools.ietf.org/html/rfc4180#page-2) says: "The last record in the file may or may not have an ending line break." So our code should be able to live with both. A canonical file can be achieved in several ways, see ​http://www.cyberciti.biz/faq/howto-linux-unix-command-remove-all-blank-lines/ .

Note: As for GML (probably not affected here), whitespace is irrelevant anyway.

[Piero Campalani]

As Dimitar you probably suggestes, I guess we can just keep current behaviour (no EOL), being it allowed by RFC standard, and just change the systemtest to use 'diff -b'.

Ok?

[Dimitar Misev]

Yes sounds good.

[Piero Campalani]

Done in changeset:17490d6.