Need detailed examples for Customize query execution and Customize the pipeline implementation

[thanhquanse]

Dear authors and all,

I am very interested in this library since it supports execution customization to help me intervene in intermediate results to build something in the middle. However, I don't know how to use the customization for the below successfully.

Can you please make a detailed example?

I appreciate your support!

[Callidon]

Hello 👋
You can look at the internals of the package as everything is based around this sytem: all base sparql operations are implemented as pipeline stage and plugged by default using builder.use when creating a new builder. It happens here https://github.com/Callidon/sparql-engine/blob/master/src/engine/plan-builder.ts#L124-L147

You can find all default stages here https://github.com/Callidon/sparql-engine/tree/master/src/engine/stages
A very simple one is the UnionStageBuilder which does an union between two inputs stages https://github.com/Callidon/sparql-engine/blob/master/src/engine/stages/union-stage-builder.ts

Note that all stages uses the Pipeline abstraction to have an abstract API to manipulate dataflows. You need to implement custom stages using the same API. The typescript defintion of this API is here https://github.com/Callidon/sparql-engine/blob/master/src/engine/pipeline/pipeline-engine.ts

I hope this help you

[thanhquanse]

Thank you very much @Callidon
I cannot believe that you replied to me very quickly with valuable and detailed instructions. I hope this sparql-engine can still be developed and supported more...
By the way, I also read some of your papers regarding RDF and SPARQL, and I am really impressed and interested in them because they are what I have been searching for so long. Therefore, I tried to send emails to you via the posted address on https://sites.google.com/view/thomas-minier/home?authuser=0 and the "Nantes" email address on your papers, but I failed with the "not existing address" error. I am not sure about the reason.

Thank you very much, @Callidon !

[Callidon]

@thanhquanse Thanks for your interest in the project. I no longer work for the university of nantes, so my email address is dead. I also no longer work on RDF/SPARLQ related projects, but I still maintain a bit this repository when Ihave some spare time.

[thanhquanse]

Thank you @Callidon
One last question. For example, with the class below, I just want to intervene in the results of "FilterStage" to do some other calculation, and I still want to use the default "Filter" implementation. However, I don't know how to achieve this properly for each stage.
I also tried to use "VectorPipeline", it is still not successful, and very hard to extract the intermediate results of each stage.

Could you please give me some advice?

Thanks a lot, @Callidon

`class MyFilterStageBuilder extends stages.FilterStageBuilder {
constructor() {
super();
}
execute(source, filterNode, customFunctions, context) {
const superFilter = super.execute(source, filterNode, customFunctions, context);
superFilter.forEach(x => {
console.log(x);
})
return superFilter;
}
}

class MyEngine extends VectorPipeline {
constructor() {
super();
}
}`