MongoDB -> Mysql Migration error: ValueError: invalid literal for int() with base 10

[feoh]

We hit this almost immediately attempting to test migrate a course in RC from MongoDB to Mysql.

Sentry issue: https://mit-office-of-digital-learning.sentry.io/share/issue/3357e48cad0e454daba03499e1d472cc/

[taimoor-ahmed-1]

@feoh it seems that the thread id is from mongodb rather than mysql, even after the migration. Can you confirm if you are hitting a fixed url which is breaking with the mentioned error OR whether you're going to the discussion tab from the course page that redirects you to this broken page?

[sarina]

@pdpinch - could you reply to @taimoor-ahmed-1 above? If this is a release blocker we need to resolve this asap

[feoh]

I reached out to @blarghmatey who conducted the test. It doesn't seem sensical to me to conduct a migration test with a dirty database but worth asking in any case! I'll post the results here as soon as I get a response.

[feoh]

Tobias and I will be working this issue tomorrow in the office.

[feoh]

Hi there, Tobias and I replicated the issue by performing the following detailed steps:

• Pick a course
  • Course chosen:https://courses-qa.mitxonline.mit.edu/courses/course-v1:TestXt+multipleruns101+1T2025/discussion/forum/course/threads/68092d5570514900
• If there's no comments in it, make some.
  - We successfully created comments and responses.
• Flip the toggle to tell it to use Forums V2
• Verify that all those comments are still present
  - We did this. Comments and responses were still present.
• Add more comments
• Run the migration - lms management command - needs course id
• Verify we can still see all the comments
  - Migration ran successfully. Output attached.
• Add more comments and make sure they end up in mysql
  - When we added additional comments and responses we hit the same exception previously reported - sentry link.
  - We can reproduce the exception repeatedly by hitting reload

[feoh]

Migration output:

admin@ip-10-21-2-183:/etc/docker/compose$ sudo COMPOSE_PROFILES=web docker compose exec -it lms /bin/bash
WARN[0000] /etc/docker/compose/docker-compose.yaml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
app@94050e09a1ee:~/edx-platform$
app@94050e09a1ee:~/edx-platform$ ./manage.py lms forum_migrate_course_from_mongodb_to_mysql course-v1:TestXt+multipleruns101+1T2025
2025-04-23 18:22:39,188 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'coursewarehistoryextended.StudentModuleHistoryExtended' instead.
  warnings.warn(

2025-04-23 18:22:39,498 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'completion.BlockCompletion' instead.
  warnings.warn(

2025-04-23 18:22:39,501 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/edx-platform/openedx/core/storage.py:114: RemovedInDjango51Warning: django.core.files.storage.get_storage_class is deprecated in favor of using django.core.files.storage.storages.
  return get_storage_class(storage_class)(**kwargs)

2025-04-23 18:22:39,704 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/pgpy/constants.py:5: DeprecationWarning: 'imghdr' is deprecated and slated for removal in Python 3.13
  import imghdr

2025-04-23 18:22:39,728 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'celery_utils.FailedTask' instead.
  warnings.warn(

2025-04-23 18:22:39,832 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'grades.PersistentSubsectionGrade' instead.
  warnings.warn(

2025-04-23 18:22:39,834 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'grades.PersistentCourseGrade' instead.
  warnings.warn(

2025-04-23 18:22:39,893 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/user_tasks/conf.py:40: RemovedInDjango51Warning: django.core.files.storage.get_storage_class is deprecated in favor of using django.core.files.storage.storages.
  return get_storage_class(import_path)()

2025-04-23 18:22:40,487 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'experiments.ExperimentData' instead.
  warnings.warn(

2025-04-23 18:22:40,560 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/edxval/utils.py:158: RemovedInDjango51Warning: django.core.files.storage.get_storage_class is deprecated in favor of using django.core.files.storage.storages.
  return get_storage_class(

2025-04-23 18:22:40,563 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/edxval/utils.py:183: RemovedInDjango51Warning: django.core.files.storage.get_storage_class is deprecated in favor of using django.core.files.storage.storages.
  return get_storage_class(

2025-04-23 18:22:40,679 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'learning_sequences.CourseSection' instead.
  warnings.warn(

2025-04-23 18:22:40,703 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/integrated_channels/utils.py:28: RemovedInDjango50Warning: The django.utils.timezone.utc alias is deprecated. Please update your code to use datetime.timezone.utc instead.
  UNIX_EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc)

2025-04-23 18:22:40,709 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'integrated_channel.ContentMetadataItemTransmission' instead.
  warnings.warn(

2025-04-23 18:22:40,710 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'integrated_channel.OrphanedContentTransmissions' instead.
  warnings.warn(

2025-04-23 18:22:40,715 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'degreed.DegreedLearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:40,719 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'degreed2.Degreed2LearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:40,739 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'sap_success_factors.SapSuccessFactorsLearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:40,744 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'cornerstone.CornerstoneLearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:40,747 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'xapi.XAPILearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:40,753 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'blackboard.BlackboardLearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:40,758 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'canvas.CanvasLearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:40,762 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/db/models/options.py:210: RemovedInDjango51Warning: 'index_together' is deprecated. Use 'Meta.indexes' in 'moodle.MoodleLearnerDataTransmissionAudit' instead.
  warnings.warn(

2025-04-23 18:22:41,451 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/drf_yasg/views.py:84: DeprecationWarning: SwaggerJSONRenderer & SwaggerYAMLRenderer's `format` has changed to not include a `.` prefix, please silence this warning by setting `SWAGGER_USE_COMPAT_RENDERERS = False` in your Django settings and ensure your application works (check your URLCONF and swagger/redoc URLs).
  warnings.warn(

2025-04-23 18:22:41,472 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/lti_consumer/lti_1p3/extensions/rest_framework/serializers.py:121: RemovedInDjango50Warning: The django.utils.timezone.utc alias is deprecated. Please update your code to use datetime.timezone.utc instead.
  timestamp = serializers.DateTimeField(input_formats=[ISO_8601], format=ISO_8601, default_timezone=timezone.utc)

2025-04-23 18:22:41,473 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/lti_consumer/lti_1p3/extensions/rest_framework/serializers.py:244: RemovedInDjango50Warning: The django.utils.timezone.utc alias is deprecated. Please update your code to use datetime.timezone.utc instead.
  default_timezone=timezone.utc,

2025-04-23 18:22:41,473 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/lti_consumer/lti_1p3/extensions/rest_framework/serializers.py:250: RemovedInDjango50Warning: The django.utils.timezone.utc alias is deprecated. Please update your code to use datetime.timezone.utc instead.
  default_timezone=timezone.utc,

2025-04-23 18:22:41,865 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/edx_sysadmin/templatetags/sysadmin_extras.py:8: RemovedInDjango50Warning: The django.utils.timezone.utc alias is deprecated. Please update your code to use datetime.timezone.utc instead.
  from django.utils.timezone import utc as UTC  # noqa: N812

2025-04-23 18:22:41,956 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/django/conf/__init__.py:201: RemovedInDjango51Warning: The STATICFILES_STORAGE setting is deprecated. Use STORAGES instead.
  self._show_deprecation_warning(

System check identified some issues:

WARNINGS:
embargo.GlobalRestrictedCountry.country: (fields.W342) Setting unique=True on a ForeignKey has the same effect as using a OneToOneField.
        HINT: ForeignKey(unique=True) is usually better served by a OneToOneField.
Migrating data for course: course-v1:TestXt+multipleruns101+1T2025
2025-04-23 18:22:42,280 INFO 7263 [elasticsearch] base.py:240 - PUT https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comment_threads/_doc/206 [status:201 request:0.059s]
2025-04-23 18:22:42,282 WARNING 7263 [py.warnings] warnings.py:110 - /openedx/.local/lib/python3.11/site-packages/elasticsearch/connection/http_urllib3.py:275: DeprecationWarning: HTTPResponse.getheaders() is deprecated and will be removed in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.
  return response.status, response.getheaders(), raw_data

2025-04-23 18:22:42,282 INFO 7263 [forum.search.es] es.py:130 - Document 206 indexed in comment_threads
2025-04-23 18:22:42,282 INFO 7263 [forum.handlers] handlers.py:138 - CommentThread 206 added to the search backend
2025-04-23 18:22:42,381 INFO 7263 [elasticsearch] base.py:240 - PUT https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_doc/91 [status:201 request:0.050s]
2025-04-23 18:22:42,382 INFO 7263 [forum.search.es] es.py:130 - Document 91 indexed in comments
2025-04-23 18:22:42,382 INFO 7263 [forum.handlers] handlers.py:138 - Comment 91 added to the search backend
2025-04-23 18:22:42,440 INFO 7263 [elasticsearch] base.py:240 - POST https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_update/91 [status:200 request:0.044s]
2025-04-23 18:22:42,440 INFO 7263 [forum.search.es] es.py:95 - Document 91 in index comments updated successfully.
2025-04-23 18:22:42,441 INFO 7263 [forum.handlers] handlers.py:141 - Comment 91 updated in the search backend
2025-04-23 18:22:42,507 INFO 7263 [elasticsearch] base.py:240 - PUT https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_doc/92 [status:201 request:0.039s]
2025-04-23 18:22:42,507 INFO 7263 [forum.search.es] es.py:130 - Document 92 indexed in comments
2025-04-23 18:22:42,507 INFO 7263 [forum.handlers] handlers.py:138 - Comment 92 added to the search backend
2025-04-23 18:22:42,577 INFO 7263 [elasticsearch] base.py:240 - POST https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_update/92 [status:200 request:0.058s]
2025-04-23 18:22:42,577 INFO 7263 [forum.search.es] es.py:95 - Document 92 in index comments updated successfully.
2025-04-23 18:22:42,577 INFO 7263 [forum.handlers] handlers.py:141 - Comment 92 updated in the search backend
2025-04-23 18:22:42,644 INFO 7263 [elasticsearch] base.py:240 - PUT https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_doc/93 [status:201 request:0.040s]
2025-04-23 18:22:42,644 INFO 7263 [forum.search.es] es.py:130 - Document 93 indexed in comments
2025-04-23 18:22:42,644 INFO 7263 [forum.handlers] handlers.py:138 - Comment 93 added to the search backend
2025-04-23 18:22:42,694 INFO 7263 [elasticsearch] base.py:240 - POST https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_update/93 [status:200 request:0.039s]
2025-04-23 18:22:42,694 INFO 7263 [forum.search.es] es.py:95 - Document 93 in index comments updated successfully.
2025-04-23 18:22:42,694 INFO 7263 [forum.handlers] handlers.py:141 - Comment 93 updated in the search backend
2025-04-23 18:22:42,767 INFO 7263 [elasticsearch] base.py:240 - PUT https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_doc/94 [status:201 request:0.041s]
2025-04-23 18:22:42,768 INFO 7263 [forum.search.es] es.py:130 - Document 94 indexed in comments
2025-04-23 18:22:42,768 INFO 7263 [forum.handlers] handlers.py:138 - Comment 94 added to the search backend
2025-04-23 18:22:42,816 INFO 7263 [elasticsearch] base.py:240 - POST https://vpc-opensearch-mitxonline-qa-3qlf4tarjsdsmfvvcet557jxw4.us-east-1.es.amazonaws.com:443/comments/_update/94 [status:200 request:0.037s]
2025-04-23 18:22:42,817 INFO 7263 [forum.search.es] es.py:95 - Document 94 in index comments updated successfully.
2025-04-23 18:22:42,817 INFO 7263 [forum.handlers] handlers.py:141 - Comment 94 updated in the search backend
Enabled mysql backend waffle flag for course course-v1:TestXt+multipleruns101+1T2025.
Data migration completed successfully

[feoh]

@sarina @taimoor-ahmed-1 Hope the above gives you the detail you need. If it doesn't let me know and I'll be sure to get back to you more quickly now that I'm personally up to speed on the black magic invocations required to test.

[taimoor-ahmed-1]

@feoh can we hop on a call today to see this issue on your end?

We're unable to open the sentry link that you have shared because of MIT org access restrictions.

[blarghmatey]

Chris is out today but I can walk you through the bug. I'll follow up on slack to coordinate a time.

[feoh]

it’s the exact same exception we got before. I’m sorry I probably didn’t share the link correctly.

I’m on vacation today and not near a computer, but @blarghmatey may be able to help in my absence.