-
Notifications
You must be signed in to change notification settings - Fork 9
logtime function exception safe #279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| Field<Long> asField() { | ||
| final String unixTimestamp = "UNIX_TIMESTAMP(STR_TO_DATE(" | ||
| + "IFNULL(SUBSTRING(REGEXP_SUBSTR({0},'^\\\\d{4}\\\\/\\\\d{2}-\\\\d{2}\\\\/[\\\\w\\\\.-]+\\\\/([^\\\\p{Z}\\\\p{C}]+?)\\\\/([^\\\\p{Z}\\\\p{C}]+)(-@)?(\\\\d+|)-(\\\\d{4}\\\\d{2}\\\\d{2}\\\\d{2})'), -10, 10), '1970010100'), '%Y%m%d%H'))"; | ||
| return DSL.field("COALESCE(" + unixTimestamp + ", 0)", Long.class, pathField); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
making 1970 as a magick value is not really an option, better way would be to return a "sql null" in case extraction is not possible.
using
SELECT REGEXP_REPLACE( '2010/01-08/sc-99-99-14-40/f17_v2/f17_v2.logGLOB-2010011601.log.gz','^\\d{4}\\/\\d{2}-\\d{2}\\/[\\w\\.-]+\\/([^\\p{Z}\\p{C}]+?)\\/([^\\p{Z}\\p{C}]+)(-@)?(\\d+|)-(\\d{4}\\d{2}\\d{2}\\d{2}).*', '\\5' );
REGEXP_REPLACE( '2010/01-08/sc-99-99-14-40/f17_v2/f17_v2.logGLOB-2010011601.log.gz','^\\d{4}\\/\\d{2}-\\d{2}\\/[\\w\\.-]+\\/([^\\p{Z}\\p{C}]+?)\\/([^\\p{Z}\\p{C}]+)(-@)?(\\d+|)-(\\d{4}\\d{2}\\d{2}\\d{2}).*', '\\5' );for example would provide better results. we should rather check that if extraction was successful and that it parses to date and then to unix date.
these can be verified with if(value regex 'valid value', unixtime(value), null) style
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
refactored to check for valid timestamp, otherwise returning SQL null
5ebd3c3 to
fa51ffc
Compare
|
rebased |
Tiihott
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logtime is handled as sql null when the value extraction is not possible. Tests pass and seem fine.
LGTM
fa51ffc to
fe78746
Compare
|
rebased |
|
added |
0c6b07d to
6a0315f
Compare
|
rebased |
…tead of throwing mariadb exception
…rmatting, improve javadoc
6a0315f to
8566092
Compare
|
rebased |
Tiihott
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Description
Logtime function throws exception in case functions fail, forcing the SQL engine to be run in a throw no exceptions mode.
This change aims to allow the logtime function to use SQL null when parsing is not possible.