xstream-1_4_21

xstream.md

There seems to be some trouble when resizing a Chronicle Map after going from XStream 1.4.20 to 1.4.21 (which is security driven release) for non-Serializable objects.

Current Chronicle map still uses xstream 1.4.20: https://github.com/OpenHFT/Chronicle-Map/blob/343bb22247c17da12ee63b4099099c085fa0cf5f/pom.xml#L132-L137 so they have not found about the issue yet.

            map.getAll(tempFilePath.toFile());

            String field = map.name();

            map.close();

            Files.delete(chronicleMapFile.toPath());

            ChronicleMap<BytesRef, Integer> m = ChronicleMap.of(BytesRef.class, Integer.class)
                    .name(field)
                    .averageKeySize(newMapAvgKey)
                    .entries(newMapSize)
                    .keyReaderAndDataAccess(BytesRefSizedReader.INSTANCE, new BytesRefDataAccess())
                    .createPersistedTo(chronicleMapFile);
            m.putAll(tempFilePath.toFile());  // throws ConversionException

com.thoughtworks.xstream.converters.ConversionException: unable to convert node named=org.apache.lucene.util.BytesRef
---- Debugging information ----
message             : unable to convert node named=org.apache.lucene.util.BytesRef
class               : net.openhft.chronicle.map.VanillaChronicleMap
required-type       : net.openhft.chronicle.map.VanillaChronicleMap
converter-type      : net.openhft.xstream.converters.VanillaChronicleMapConverter
line number         : -1
version             : 1.4.21
-------------------------------

The format of the JSON file written by getAll() has not changed. Looks like this:

{
  "cmap": [
    {
      "entry": [
        {
          "org.apache.lucene.util.BytesRef": {
            "bytes": [
              "NA=="
            ],
            "offset": 0,
            "length": 1
          },
          "java.lang.Integer": 4
        },
        {
          "org.apache.lucene.util.BytesRef": {
            "bytes": [
              "Mg=="
            ],
            "offset": 0,
            "length": 1
          },
          "java.lang.Integer": 2
        },
        ...

1.4.20

The reader object is StaxReader / AbstractPullReader

public Object unmarshal(HierarchicalStreamReader reader, UnmarshallingContext context) {  // cmap
        if ("[\"\"]".equals(reader.getValue())) {
            return null;
        } else if (!"cmap".equals(reader.getNodeName())) {
            throw new ConversionException("should be under 'cmap' node");
        } else {
            reader.moveDown();     // cmap -> cmap

            for(; reader.hasMoreChildren(); reader.moveUp()) {
                reader.moveDown();    // cmap -> entry
                String nodeName0 = reader.getNodeName();
                if (!nodeName0.equals("entry")) {
                    throw new ConversionException("unable to convert node named=" + nodeName0);
                }

The first call to reader.moveDown(); will stay at cmap.

There is no call to the unmarshall strategy in XStream !

1.4.21

The reader object is StaxReader / AbstractPullReader, like in 1.4.20

There is extra call to marshallingStrategy in XStream:

public Object unmarshal(HierarchicalStreamReader reader, Object root, DataHolder dataHolder) {
        try {
            if (this.collectionUpdateLimit > 0) {
                if (dataHolder == null) {
                    dataHolder = new MapBackedDataHolder();
                }

                dataHolder.put("XStreamCollectionUpdateLimit", new Integer(this.collectionUpdateLimit));
                dataHolder.put("XStreamCollectionUpdateSeconds", new Integer(0));
            }

            try {
                return this.marshallingStrategy.unmarshal(root, reader, dataHolder, this.converterLookup, this.mapper);
            }

Upon entry to unmarshall() the reader.getNodeName() is cmap. The first call to reader.moveDown(); moves to entry and the first in the cycle moves to BytesRef, causing the ConversionException exception to be thrown.

public Object unmarshal(HierarchicalStreamReader reader, UnmarshallingContext context) {  // cmap
        if ("[\"\"]".equals(reader.getValue())) {
            return null;
        } else if (!"cmap".equals(reader.getNodeName())) {
            throw new ConversionException("should be under 'cmap' node");
        } else {
            reader.moveDown();    // cmap -> entry

            for(; reader.hasMoreChildren(); reader.moveUp()) {
                reader.moveDown();   // entry -> org.apache.lucene.util.BytesRef
                String nodeName0 = reader.getNodeName();
                if (!nodeName0.equals("entry")) {
                    throw new ConversionException("unable to convert node named=" + nodeName0);    // !!!!!!!!
                }

AbstractPullReader:

public void moveDown() {
        int currentDepth = this.elementStack.size();

        while(this.elementStack.size() <= currentDepth) {
            this.move();
            if (this.elementStack.size() < currentDepth) {
                throw new RuntimeException();
            }
        }

    }