Opened 4 years ago

Closed 4 years ago

#551 closed defect (fixed)

WCPS responses do not add EOL at the end of the response

Reported by: pcampalani Owned by: pcampalani
Priority: trivial Milestone: 9.0
Component: petascope Version: development
Keywords: eol wcps Cc: dmisev, pbaumann
Complexity: Medium

Description

newline (\n, 0x0a) should be put at the end of a CSV encoded WCPS response.

By default, vim automatically adds it on save, so systemtests can become unreliable when oracles are manually edited.

This currently inhibits multipoint WCPS tests to pass.

$ echo `pwd`
/home/rasdaman/rasdaman/systemtest/testcases_services/test_wcps
$ hexdump -C output/62-subset-all-dims-multipoint.test.out | tail -2
00000800  2c 32 35 35 2c 32 35 35  7d                       |,255,255}|
00000809
$ hexdump -C oracle/62-subset-all-dims-multipoint.test.oracle | tail -2
00000800  2c 32 35 35 2c 32 35 35  7d 0a                    |,255,255}.|
0000080a
$ hexdump -C oracle/01-just_trim.test.oracle | tail -2
00000130  49 45 4e 44 ae 42 60 82                           |IEND.B`.|
00000138

Change History (10)

comment:1 Changed 4 years ago by pcampalani

  • Solution 1: CSVs with EOL at the end of the response
    • fix Petascope
  • Solution 2: CSVs without EOL at the end of the response
    • fix multipoint-related templates (remove EOL)

comment:2 Changed 4 years ago by dmisev

No WCPS should not add EOL in my opinion, it's better to fix the tests to match the output.

comment:3 Changed 4 years ago by pcampalani

I don't have particular preferences, I just thought it was good practice for text files and internet protocols.

Why would you keep no-eol?

comment:4 Changed 4 years ago by dmisev

  • Cc pbaumann added

I think you'd want to end the line with a new line when the next line holds more information, and is not just an empty line that is the end of output?

Peter do you maybe have any opinion on this, should scalar output (or csv encoding) from WCPS end with a new line?

comment:5 Changed 4 years ago by dmisev

We could modify the diff command in the systemtest as well to ignore whitespace, I think we don't have any output that is critical of whitespace (like Python code for example).

comment:6 Changed 4 years ago by dmisev

It's an issue of semantics, e.g. from wikipedia:

Two ways to view newlines, both of which have valid internal logic, are that newlines terminate lines or that they separate lines. If a newline is considered a separator, there will be no newline after the last line of a file.

We don't have multiline output from WCPS (?) so in that sense it would be valid for the output to end with a \n

Last edited 4 years ago by dmisev (previous) (diff)

comment:7 Changed 4 years ago by pbaumann

I understand that this affects only CSV? Its definition in RFC 4180 (http://tools.ietf.org/html/rfc4180#page-2) says: "The last record in the file may or may not have an ending line break." So our code should be able to live with both. A canonical file can be achieved in several ways, see http://www.cyberciti.biz/faq/howto-linux-unix-command-remove-all-blank-lines/ .

Note: As for GML (probably not affected here), whitespace is irrelevant anyway.

comment:8 Changed 4 years ago by pcampalani

As Dimitar you probably suggestes, I guess we can just keep current behaviour (no EOL), being it allowed by RFC standard, and just change the systemtest to use 'diff -b'.

Ok?

comment:9 Changed 4 years ago by dmisev

Yes sounds good.

comment:10 Changed 4 years ago by pcampalani

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.