Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# Xtreamer

[![npm version][npm-image]][npm-url]
Expand Down Expand Up @@ -115,7 +114,7 @@ xtreamerTransform.on("data", (data) => { });

### max_xml_size ( default - 10000000 )

`max_xml_size` is maximum the number of characters allowed to hold in memory.
`max_xml_size` is maximum the number of characters allowed to hold in memory.

`xtreamer` raises an `error` event in case in memory xml string exceed specified limit. This ensures that the node process doesn't get terminated because of excess in memory data collection.

Expand Down Expand Up @@ -145,6 +144,34 @@ Note that the converted JSON is internally stringified before sending it back to

In case transformer function encounters an error, `xtreamer` emits `error` event and stops the xml conversion process.

## pass_all_nodes (default - false)

`pass_all_nodes` defines whether to pass only matching nodes or all of them.

This can be helpful when modifying an XML document on the fly without loosing non matching nodes.

```javascript
const xtreamer = require("xtreamer");
// Override `pass_all_nodes`
const options = { pass_all_nodes: true };
const xtreamer = Xtreamer("XmlNode", options)
```

The receiving function must verify before processing, i.e.
```javascript
// A piped transformer
async _transform(chunk, enc, done) {
if (Buffer.isBuffer(chunk)) chunk = chunk.toString();
if (!chunk.startsWith("<XmlNode")) {
// Not interesting, just pipe out
this.push(chunk);
}
// Do something with an interesting matching node as usual
this.push(chunk + "\n");
done();
}
```

## Usage

Following code snippet uses `request` NPM package as input readable stream -
Expand Down
16 changes: 15 additions & 1 deletion lib/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ class Xtreamer extends Transform {
}

_flush(done) {
if (this._options.pass_all_nodes) {
this.push(this._xmlString.trim());
}
done();
}

Expand All @@ -31,10 +34,21 @@ class Xtreamer extends Transform {
}
const nodeIndices = getNodeIndices(this._xmlString, this._node);
let nodes = [];
let fromIndex = 0;

for (let index = 0; index < nodeIndices.length; index++) {
const nodeObj = nodeIndices[index];
const endIndex = nodeObj.end + this._node.length + 3;
const xmlNode = this._xmlString.slice(nodeObj.start, endIndex);

if (this._options.pass_all_nodes) {
const otherNode = this._xmlString.slice(fromIndex, nodeObj.start).trim();
fromIndex = endIndex;
if (otherNode.length > 0) {
this.push(otherNode);
nodes.push(otherNode);
}
}
this._options && this._options.transformer && typeof this._options.transformer === "function" ?
this.push(JSON.stringify(await this._options.transformer(xmlNode))) :
this.push(xmlNode);
Expand All @@ -52,4 +66,4 @@ module.exports = (node, options = { max_xml_size: Constants.MAX_XML_LENGTH }) =>
options = options || {};
options.max_xml_size = options.max_xml_size || Constants.MAX_XML_LENGTH;
return new Xtreamer(node, options);
};
};
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "xtreamer",
"version": "1.1.2",
"version": "1.1.3",
"description": "A NodeJS package to read XML files using NodeJS streams.",
"main": "lib/index.js",
"scripts": {
Expand Down