This repository was archived by the owner on Jul 5, 2025. It is now read-only.
First request for Mongo FDW #26
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also added code for outputing BSON dates to "MongoDB Extended JSON" [1] [1] http://docs.mongodb.org/manual/reference/mongodb-extended-json/
This will avoid multiple connection and disconnection to MongoDB Server. FDW's Options functions separated from "mongo_fdw.c" and placed in "option.c" file. The newly created connection's pooling functions placed into "connection.c" file.
causing the server to crash while query is used in any container, i.e in PlpgSQl function.
Conflicts: Makefile README.md mongo_fdw.c mongo_query.c
Reported on GitHub through issue #58 by simon-wolf. FDW-631, Vaibhav Dalvi, reviewed by Jeevan Chalke.
Code changes involve adjusting the make_restrictinfo() call per PostgreSQL's upstream changes. Matches reltarget entries only on varno/varattno. It also takes into account the new RTEPermissionInfo stuff. Beginning with v16, fs_relids includes the rangetable indexes of outer joins, if any were involved in this join. The new field fs_base_relids includes only base relation indexes and thus mimics fs_relids's old semantics. Need to adjust the fdw code to account for this. FDW-603, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajay Pal, a few changes by Jeevan Chalke.
I forgot to include them in the previous commit, i.e. 0919481. FDW-603, Vaibhav Dalvi.
FDW-626, Vaibhav Dalvi, reviewed by Jeevan Chalke.
The execution of mongodb_init.sh is compulsory before running regression, and it is missed to mention it in the README file. Also, update a few comments in test cases. FDW-600, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke.
MongoDB stopped supporting this driver version for a few years now, and we have already given deprecation warnings in the previous releases. So, clean up the related code. FDW-664, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajay Pal.
The main changes include moving the installation steps to the newly created INSTALL.md file. Other changes in README.md are for unifying it with FDW documentation template. Reported on GitHub through issue #168 by mkgrgis (Михаил). FDW-665, initial patch by mkgrgis, further revised and improved by Vaibhav Dalvi.
Tests involving MIN()/MAX() aggregates sometimes take a shortcut path due to hardware differences, resulting in the explain plan changes and, thus causing regression failures. To get a consistent result, change those to SUM()/AVG() aggregates or add a relevant grouping clause. The patch also tweaks the startup cost for the ordered paths so that ORDER BY will get pushed for a few more queries. FDW-598, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, tested by Ajay Pal.
FDW-676, Suraj Kharage.
Standard Support for both EDB Postgres Advanced Server 11 and PostgreSQL 11 is already ended. Thus, adjust Makefile so that we restrict compilation of mongo_fdw code against v11. Update the README accordingly. Also, clean up the code for the same. FDW-673, Suraj Kharage, reviewed by Sravan Velagandula, tested by Ajaykumar Pal.
Code changes involve adjusting calls to some server-side functions due to their signature changes. FDW-682, Suraj Kharage, reviewed by Jeevan Chalke.
This allows building the extension from the source's contrib directory without the need for pg_config availability in the path. Jeevan Chalke.
FDW-709, Vaibhav Dalvi, tested by Kashif Zeeshan.
The variable name we use to declare the column of outer relation
using the $let field is the same as the column name. However, if
the column is a sub-column (i.e., "parent.child"), then the variable
name also contains the dot ("."), which is not allowed in variable
name formation, resulting in a wrong resultset being returned from
the server.
Since special characters other than underscore ("_") are not allowed,
let's use underscore instead of dot to fix the issue.
Reported on GitHub through issue #173 by Ömer Sezgin Uğurlu (ugurlu).
FDW-669, Vaibhav Dalvi, reviewed by Sravan Velagandula,
tested by Kashif Zeeshan.
Vaibhav Dalvi.
Standard support has ended for EDB Postgres Advanced Server 12, Postgres Extended Server 12, and PostgreSQL 12. This commit removes compatibility with v12 by updating the Makefile to prevent building mongo_fdw against it. Corresponding updates are made to the README, and obsolete code paths related to v12 are cleaned up. FDW-716, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajaykumar Pal.
Pushable WHERE clause expressions were incorrectly added to the MongoDB query pipeline using bsonAppend[Start/Finish]Array, leading to the warning: "bson_append_array(): invalid array detected". This commit corrects the placement of the filter clause in the pipeline to avoid the error and ensure proper query construction. Reported on GitHub through issue #188 by Chris Williams (C-Williams). DB-720, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajaykumar Pal.
In join queries, the result is correctly labeled as "Join_Result". However, while constructing the sort pipeline for the ORDER BY clause, the column reference mistakenly used "Join_result" (with incorrect casing). This mismatch prevented the expected row ordering. This commit corrects the name usage to ensure proper ORDER BY pushdown. DB-721, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajaykumar Pal.
Ensure the buffer size is sufficient to safely hold a string representation of an integer, including its sign. Replace sprintf() with snprintf() to avoid potential buffer overflows and improve safety. DB-722, Vaibhav Dalvi, reviewed by Suraj Kharage.
Upgrade the libmongoc driver to version 1.30.2 and the json-c library to version 0.18-20240915. Deprecated APIs have been replaced with their modern equivalents as part of this update. The previous versions had limitations that prevented reliable connections to newer MongoDB servers, particularly version 8. With this change, MongoDB 8 is now supported. Compatibility with MongoDB versions 5 and earlier may be affected, but those versions are already end-of-life (EOL). FDW-713, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Kashif Zeeshan.
EDB Postgres Extended Server 18. Code changes include updating calls to certain server-side functions to match their modified signatures, along with adjustments to related constants. FDW-725, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan.
Introduce a new GUC log_remote_query to enable logging of the remote query that would be executed on the MongoDB server. This is intended for debugging purposes only. When enabled, the remote query is printed to the PostgreSQL server logs. The default value is false. FDW-729, Vaibhav Dalvi, reviewed and some adjustments by Jeevan Chalke.
I failed to add in 3333079. Reported by Vaibhav Dalvi.
Replace sprintf() with snprintf() to prevent potential buffer overflows and enhance code safety. Also, remove the strlen() call by using the length returned by bson_as_relaxed_extended_json(). Vaibhav Dalvi, reviewed by Jeevan Chalke.
…lauses. Previously, join pushdown was restricted when full document retrieval was involved in the target list, due to the lack of infrastructure for correctly retrieving and processing full documents locally from join results. However, this restriction was not consistently applied to restrict clauses (WHERE and JOIN clauses). This oversight could lead to incorrect results when such conditions were evaluated locally. This commit extends the existing restriction to prevent join pushdown when full document retrieval is involved in either the target list or any restrict clause. For similar reasons, this commit also restricts join pushdown when whole row references are present in restrict clauses. These changes ensure correctness by preventing scenarios where local evaluation cannot properly handle complex data types or complete row structures derived from pushed-down joins. Reported on GitHub through issue #191 by ephemeralctl. FDW-733, Vaibhav Dalvi and Jeevan Chalke.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
14 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First request for Mongo FDW