Write git commands which are run to the log

[eudoxos]

Hi, I am still experiencing issues with the plugin, but I am very limited in finding out what's wrong (and submitting better reports) as all I see in the log is:

I, [2016-07-30T09:21:25.205230 #12612]  INFO -- :   Parameters: {"project_id"=>"brick-micro"}
E, [2016-07-30T09:21:25.375670 #12612] ERROR -- : Availability check failed due to failed Git command: git exited with non-zero status: 128

but the log does not show what git command was being run and failed. Could that be added to the log, please? Then I can try running the command from command-line and check the error output (which could be also part of the log, perhaps). Thanks!

[mabentwickeltsich]

Hey @eudoxos,

I will give you a new test version of the plugin printing the Git command that is being executed. Perhaps tomorrow morning. Sorry for the delay.

Cheers
Luis

[mabentwickeltsich]

Hey @eudoxos,

I had a look at the problem and the code that raises the error is part of OpenProject, not directly in the plugin. It is the method check_availability! of the class Git < Base (openproject/lib/open_project/scm/adapters/git.rb, around line 84 in OP 5.0.13). The first line of this method is the culprit:

  out, = Open3.capture2e(client_command, *build_git_cmd(%w[log -- HEAD]))

I suggest you to edit the file and add a few extra lines of code to print the command being executed. Then you can test it manually to check what error message the command returns.

This:

        def check_availability!
          out, = Open3.capture2e(client_command, *build_git_cmd(%w[log -- HEAD]))
          [...]

should look like this:

        def check_availability!
          temp_client_command = client_command
          temp_build_git_cmd = build_git_cmd(%w[log -- HEAD])
          logger.error("Checking availability of Git repo (client_command): #{temp_client_command}")
          logger.error("Checking availability of Git repo (build_git_cmd): #{temp_build_git_cmd}")
          out, = Open3.capture2e(client_command, *build_git_cmd(%w[log -- HEAD]))
          [...]

Once you edit the file, restart OpenProject with:

touch ~/openproject-ce/tmp/restart.txt

Then have a look at the logs. With logger.error, the message will appear in the logs with any call to check_availability! (even if the command is executed successfully). With the new information we may be able to find the origin of the problem and correct the plugin if necessary.

Cheers
Luis

[eudoxos]

Hi Luis, very nice, thank you; I see this in the log:

E, [2016-08-03T19:27:44.030450 #26115] ERROR -- : Checking availability of Git repo (client_command): git
E, [2016-08-03T19:27:44.030801 #26115] ERROR -- : Checking availability of Git repo (build_git_cmd): ["--git-dir", "umicore/brick-micro.git", "-c", "core.quotepath=false", "log", "--", "HEAD"]

Not sure how to reproduce this; what is the directory git is run in (--git-dir only includes relative path)? The repo exists in /var/lib/gitolite3/repositories. ... ?

[eudoxos]

I tried with repository in a different project which works (one of the few, never knew why actually) and here, the full repo path is passed as argument (!!) thus the availability check succeeds:

E, [2016-08-03T19:56:53.714148 #26115] ERROR -- : Checking availability of Git repo (client_command): git
E, [2016-08-03T19:56:53.714396 #26115] ERROR -- : Checking availability of Git repo (build_git_cmd): ["--git-dir", "/var/lib/gitolite3/repositories/oto/caofeidian.git", "-c", "core.quotepath=false", "log", "--", "HEAD"]

[mabentwickeltsich]

@eudoxos,

I understand what the problem is. This issue is related to the upgrade of the plugin from OpenProject 4 where the paths to the repositories were relative. I need to rework this part to solve the problem, but it may take some time.
I assume that the repositories having problems were created in OpenProject 4, but those created in OpenProject 5 work fine.
What you can do in this moment is to run manually a few MySQL queries to update the paths of your repos in order to solve the problem. This is what you have to do, I included a small example:

Connect to MySQL as root:

mysql -u root -p

Select your DB (assuming the name is openproject):

mysql> USE openproject;

List of all your managed Gitolite repositories:

mysql> SELECT * FROM repositories WHERE (type = "Repository::Gitolite") AND (scm_type = "managed");
+----+------------+----------------------------------------------+-------+----------+----------------------------------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+
| id | project_id | url                                          | login | password | root_url                                     | type                 | path_encoding | log_encoding | scm_type | required_storage_bytes | storage_updated_at |
+----+------------+----------------------------------------------+-------+----------+----------------------------------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+
|  1 |          2 | /var/lib/gitolite3/repositories/test-git.git |       |          | /var/lib/gitolite3/repositories/test-git.git | Repository::Gitolite | NULL          | NULL         | managed  |                      0 | NULL               |
|  3 |          1 | demo-project.git                             |       |          | demo-project.git                             | Repository::Gitolite | NULL          | NULL         | managed  |                      0 | NULL               |
+----+------------+----------------------------------------------+-------+----------+----------------------------------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+

List of repositories without a full path (with wrong configuration):

mysql> SELECT * FROM repositories WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND ((url NOT LIKE "/var/lib/gitolite3/repositories/%") OR (root_url NOT LIKE "/var/lib/gitolite3/repositories/%"));
+----+------------+------------------+-------+----------+------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+
| id | project_id | url              | login | password | root_url         | type                 | path_encoding | log_encoding | scm_type | required_storage_bytes | storage_updated_at |
+----+------------+------------------+-------+----------+------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+
|  3 |          1 | demo-project.git |       |          | demo-project.git | Repository::Gitolite | NULL          | NULL         | managed  |                      0 | NULL               |
+----+------------+------------------+-------+----------+------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+

Adding full path to repositories with wrong configuration (separated into two queries):

mysql> UPDATE repositories SET      url = CONCAT("/var/lib/gitolite3/repositories/", url)      WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND (url NOT LIKE "/var/lib/gitolite3/repositories/%");
mysql> UPDATE repositories SET root_url = CONCAT("/var/lib/gitolite3/repositories/", root_url) WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND (root_url NOT LIKE "/var/lib/gitolite3/repositories/%");

Checking repositories without a full path (with wrong configuration) again:

mysql> SELECT * FROM repositories WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND ((url NOT LIKE "/var/lib/gitolite3/repositories/%") OR (root_url NOT LIKE "/var/lib/gitolite3/repositories/%"));
Empty set (0.00 sec)

Cheers
Luis

[eudoxos]

Hello Luis, thank you. I am mostly off internet for 2 weeks, so cannot test. May I suggest, though, that this modification of paths in the database be done as a part of the configuration recovery in plugin settings? That would seem quite a natural place for that. Not sure if just the old repos don't work, IIRC I was not able to open on I just created with OP5, but not sure now. Cheers, Vaclav

1. srpna 2016 16:57:55 GMT+05:30, "José Luis González García" notifications@github.com napsal:

   @eudoxos,

   I understand what the problem is. This issue is related to the upgrade
   of the plugin from OpenProject 4 where the paths to the repositories
   were relative. I need to rework this part to solve the problem, but it
   may take some time.
   I assume that the repositories having problems were created in
   OpenProject 4, but those created in OpenProject 5 work fine.
   What you can do in this moment is to run manually a few MySQL queries
   to update the paths of your repos in order to solve the problem. This
   is what you have to do, I included a small example:

   Connect to MySQL as root:
   mysql -u root -p

   Select your DB (assuming the name is openproject):
   mysql> USE openproject;

   List of all your managed Gitolite repositories:
   mysql> SELECT * FROM repositories WHERE (type = "Repository::Gitolite") AND (scm_type = "managed"); +----+------------+----------------------------------------------+-------+----------+----------------------------------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+ | id | project_id | url | login | password | root_url | type | path_encoding | log_encoding | scm_type | required_storage_bytes | storage_updated_at | +----+------------+----------------------------------------------+-------+----------+----------------------------------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+ | 1 | 2 | /var/lib/gitolite3/repositories/test-git.git | | | /var/lib/gitolite3/repositories/test-git.git | Repository::Gitolite | NULL | NULL | managed | 0 | NULL | | 3 | 1 | demo-project.git | | | demo-project.git | Repository::Gitolite | NULL | NULL | managed | 0 | NULL | +----+------------+----------------------------------------------+-------+----------+----------------------------------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+

   List of repositories without a full path (with wrong configuration):
   mysql> SELECT * FROM repositories WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND ((url NOT LIKE "/var/lib/gitolite3/repositories/%") OR (root_url NOT LIKE "/var/lib/gitolite3/repositories/%")); +----+------------+------------------+-------+----------+------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+ | id | project_id | url | login | password | root_url | type | path_encoding | log_encoding | scm_type | required_storage_bytes | storage_updated_at | +----+------------+------------------+-------+----------+------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+ | 3 | 1 | demo-project.git | | | demo-project.git | Repository::Gitolite | NULL | NULL | managed | 0 | NULL | +----+------------+------------------+-------+----------+------------------+----------------------+---------------+--------------+----------+------------------------+--------------------+

   Adding full path to repositories with wrong configuration (separated
   into two queries):
   mysql> UPDATE repositories SET url = CONCAT("/var/lib/gitolite3/repositories/", url) WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND (url NOT LIKE "/var/lib/gitolite3/repositories/%"); mysql> UPDATE repositories SET root_url = CONCAT("/var/lib/gitolite3/repositories/", root_url) WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND (root_url NOT LIKE "/var/lib/gitolite3/repositories/%");

   Checking repositories without a full path (with wrong configuration)
   again:
   mysql> SELECT * FROM repositories WHERE (type = "Repository::Gitolite") AND (scm_type = "managed") AND ((url NOT LIKE "/var/lib/gitolite3/repositories/%") OR (root_url NOT LIKE "/var/lib/gitolite3/repositories/%")); Empty set (0.00 sec)

   Cheers
   Luis

   ---

   You are receiving this because you were mentioned.
   Reply to this email directly or view it on GitHub:
   Write git commands which are run to the log #31 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[mabentwickeltsich]

@eudoxos

That is a good idea; I will add the functionality to correct this issue (when upgrading the plugin from OP4 to OP5) in the recovery section as soon as I have the time.

Let me know if you have solved the problem in your installation with the SQL scripts I provided. If that is the case, then I do not have to try to find more possible errors in the code.

Cheers
Luis

[eudoxos]

Hi @mabentwickeltsich, it worked, now I can all repos again. If you add this database modification to the recovery function, I am sure it will be useful for other people. Thanks for the precise instruction and help which I appreciate a lot!

I still think logging git command being run could be enabled, but if it is an upstream issue, feel free to close this ticket.