Inconsistent WebHDFS path handling over Hadoop versions

[MaicoTimmerman]

Given the following test-case:

# test_client.py
class TestAppend(_IntegrationTest):
  def test_simple_with_plus(self):
    self.client.write('ap+q', b'hello,', overwrite=True)
    self.client.write('ap+q', b' world!', append=True)
    eq_(self._read('ap+q'), b'hello, world!')

We are failing on hadoop 3.1.1 with the following error:

  File "./hdfs/test/test_client.py", line 274, in test_simple_with_plus
    self.client.write('ap+q', b' world!', append=True)
  File "./hdfs/hdfs/client.py", line 501, in write
    res = self._append(hdfs_path, buffersize=buffersize)
  File ".//hdfs/hdfs/client.py", line 118, in api_handler
    raise err
Exception: File /user/<user>/.hdfscli/ap+q not found.

and succeeding on 3.1.4, due to changes made in:

• https://issues.apache.org/jira/browse/HDFS-14423
• https://issues.apache.org/jira/browse/HDFS-14466

As upgrading HDFS is a non-trivial operation, I'm wondering if this client library would be able to special-case this inconsistency to allow for smooth upgrading over HDFS versions.