completes with errors

[thermionic]

Git pull to get the production instance up to date
Spun up a new VM for postgresql
Base setup of web server and git clone of tt-rs (so both production and new are at the same level)
Ran through the install wizard and confirmed I could login as admin
Edited the config.properties to match the production and new databases

java -Xmx4096m -jar tt-rss-mysql2pgsql.jar > output-1.txt

output-1.txt

No users are created, the table ttrss_users only has one line for the admin user

From a quick glance, the table ttrss_entries does have the expected articles in it

[hrk]

Ok, I perfectly understand the issue here.

I highly appreciated your accurate report w/ the heap requirement, the usebatch=true is both the cause of the issue and the requirement for the huge heap space (especially with huge databases).

If you run the tool with usebatch=false it will complete correctly. I am, however, preparing another release where I manually skip the admin account and the forum feed, so that usebatch=true can be used successfully.

[hrk]

Ok, this is fixed in 840b6d2, just let me know if the usebatch=true execution completes and the resulting database is just like you expect it to be.

[thermionic]

I ran it with usebatch=true

the feeds and entries are there :-)

but there is a problem, this is the preferences from the working one

and the imported one has some "duplication", if any of them are changed and then saved, they appear to save, but if I exit preferences and go back in to preferences, they go back to the values there

[hrk]

That looks very strange. That page should be built at runtime with the values read from ttrss_user_prefs.

It looks like your user has duplicate entries.

I guess the "ran through the install wizard and confirmed I could login as admin" is a step which is not required for the tool, and it's generating the extra rows (since the original ones from MySQL will be copied).

Please, drop the database, run the schema then the tool, then finally log in to verify everything is as expected.

[thermionic]

reverted to snapshot, created database and user, ran the schema .sql manually (via webmin)

java -Xmx4096m -jar tt-rss-mysql2pgsql.jar > output-1.txt

Exception in thread "main" org.postgresql.util.PSQLException: ERROR: permission denied for sequence ttrss_users_id_seq
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2182)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1911)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:173)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:615)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:465)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:458)
at it.sineo.ttrssMySQL2PGSQL.Sync.main(Sync.java:243)

reverted to snapshot, created database and user, removed the following two lines from the schema .sql

insert into ttrss_feeds (owner_uid, title, feed_url) values
(1, 'Tiny Tiny RSS: Forum', 'http://tt-rss.org/forum/rss.php');

Ran through the install wizard including initialising the database and confirmed I could login as admin

Edited the config.properties to match the production and new databases

java -Xmx4096m -jar tt-rss-mysql2pgsql.jar > output-1.txt
output-1.txt

The preferences now looks correct, but it didn't bring all articles across

[hrk]

Ok, I can see what you did wrong.

• ERROR: permission denied for sequence ttrss_users_id_seq --> the permissions/grants for the user configured in the config.properties of the tool are wrong. Most likely you're using a single (database) user, so it's his grants which are wrong/missing.

• Please follow the procedure detailed in the README.md :) especially...

• 1. Don't delete anything from the schema, it's not required. Especially in the last iteration of the tool.
• 2. Don't login as admin before launching the tool.

[thermionic]

some small changes...

did not change schema, and did not login

reverted to snapshot

created postgresql user and database

sudo -u postgres psql
CREATE USER "ttrss" WITH PASSWORD 'SooperSekritPassword';
CREATE DATABASE ttrss WITH OWNER "ttrss";
\quit

ran the ttrss schema .sql

sudo -u postgres psql -d ttrss -a -f /var/www/tt-rss/schema/ttrss_schema_pgsql.sql

reset grants

sudo -u postgres psql 
GRANT ALL ON DATABASE ttrss to ttrss;
\quit

edited the config.properties with appropriate value

ran the migrator

java -Xmx4096m -jar tt-rss-mysql2pgsql.jar > output-1.txt
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: permission denied for sequence ttrss_users_id_seq
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2182)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1911)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:173)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:615)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:465)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:458)
        at it.sineo.ttrssMySQL2PGSQL.Sync.main(Sync.java:243)

cat output-1.txt
INSERT INTO ttrss_users(id, login, pwd_hash, last_login, access_level, email, full_name, email_digest, last_digest_sent, salt, created, twitter_oauth, otp_enabled, resetpass_token) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ERROR: Batch entry 0 INSERT INTO ttrss_users(id, login, pwd_hash, last_login, access_level, email, full_name, email_digest, last_digest_sent, salt, created, twitter_oauth, otp_enabled, resetpass_token) VALUES (2, 'Arne', 'MODE2:40323aaa1d4beb3e00484b8591163376c21592b25491dc4999bcbab3beca1244', '2017-06-10 09:45:09.000000 +01:00:00', 10, 'user@domain.net', 'Arne', '0', NULL, '5fa2f9ecd4077bf106873db1c468d266da8d485a69395f0c50973ce27af242a22424f509583f9d6188910516d896422dcd5773466e3e084eb76e4c3d5c318150140cbcb4f38f409aaea44b63074a1e82e497bf6cb421fb7b235f2a34b0abcf8fac9403c1ec3135ac849d73f1f3ef5dc99ed2882bdf7bdb3b1022d23a01', '2013-08-16 19:32:41.000000 +01:00:00', NULL, '0', '1456185580:914b1c5aac882d93b95844c4fb9df2eab0eb5040') was aborted.  Call getNextException to see the cause.
CAUSE: ERROR: permission denied for relation ttrss_users
[ttrss_users] 42 ms.
[ttrss_users] setting sequence to 2

[hrk]

I am very puzzled.
The line raising the exception is when I call SELECT setval(sequenceName, lastValue) in order to update the sequence.

Google-ing around it looks like, in order to update the sequence, it is required to have GRANT USAGE ON SEQUENCE xxxx TO yyyy. I have no clue as to why your GRANT ALL ON DATABASE is not enough and why there hasn't been a report of this issue in the past (or why I haven't had it myself). What's more is that it seems to be the only sequence where the update fails, is this right?

[thermionic]

it doesn't progress after the failure, so I am not able to say if it is the only one

[cyorps]

I had no problems! Everything was running without errors and ttrss-instance is working after migration. Migration was from MariaDB version 10.3.22 to pSQL version 12.1 with Database-Version of ttrrs 139.

Thanks for this great tool!

[chrisbrannonnc]

I ran into the same problem as thermionic. Solved with:
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO ttrss_user;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO ttrss_user;

After that, though, it runs for a bit then errors out:

java -jar -Xmx3000m tt-rss-mysql2pgsql.jar

INSERT INTO ttrss_users(id, login, pwd_hash, last_login, access_level, email, email_digest, last_digest_sent, created, full_name, twitter_oauth, salt, otp_enabled, resetpass_token) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
[ttrss_users] 54 ms.
[ttrss_users] setting sequence to 2
INSERT INTO ttrss_feed_categories(id, owner_uid, title, collapsed, order_id, parent_cat, view_settings) VALUES (?, ?, ?, ?, ?, ?, ?)
[ttrss_feed_categories] 14 ms.
[ttrss_feed_categories] setting sequence to 4
INSERT INTO ttrss_feeds(id, owner_uid, title, cat_id, feed_url, icon_url, update_interval, purge_interval, last_updated, last_error, site_url, auth_login, auth_pass, parent_feed, private, rtl_content, hidden, include_in_digest, cache_images, auth_pass_encrypted, last_viewed, last_update_started, always_display_enclosures, update_method, order_id, mark_unread_on_update, strip_images, update_on_checksum_change, pubsub_state, favicon_last_checked, cache_content, hide_images, view_settings, favicon_avg_color, feed_language, last_modified, last_unconditional) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
[ttrss_feeds] 627 ms.
[ttrss_feeds] setting sequence to 144
INSERT INTO ttrss_archived_feeds(id, owner_uid, title, feed_url, site_url) VALUES (?, ?, ?, ?, ?)
[ttrss_archived_feeds] 50 ms.
INSERT INTO ttrss_counters_cache(feed_id, owner_uid, value, updated) VALUES (?, ?, ?, ?)
[ttrss_counters_cache] 246 ms.
INSERT INTO ttrss_cat_counters_cache(feed_id, owner_uid, value, updated) VALUES (?, ?, ?, ?)
[ttrss_cat_counters_cache] 7 ms.
Exception in thread "main" com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 660 milliseconds ago. The last packet sent successfully to the server was 434,759 milliseconds ago.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:400)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1038)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3434)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3334)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3774)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:871)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1940)
at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3285)
at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463)
at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3009)
at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2257)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2650)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2541)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2499)
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1432)
at it.sineo.ttrssMySQL2PGSQL.Sync.main(Sync.java:110)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 11,930 bytes, read 11,319 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2926)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3394)
... 13 more