Nested with null values, number type, and integer instead of INT

[b-rockx]

Error while compiling statement: FAILED: ParseException line 3:2 mismatched input 'mapInfo' expecting ) near 'STRING' in create table statement

Below you can see that the statement has a few issues.

1. Missing commas
2. Displaying NULLs. Maybe we could have NULLs set as STRING through an option
3. INTEGER instead of INT
4. NUMBER instead of INT

`CREATE TABLE test(
macAddress STRING
mapInfo STRUCT<
mapHierarchyDetails: STRUCT<
campus: NULL,
building: NULL,
floor: NULL,
floorAesUid: INTEGER,
zones: NULL
>

mapCoordinate STRUCT<
x: NUMBER,
y: NUMBER,
z: NUMBER,
unit: STRING

currentlyTracked BOOLEAN
confidenceFactor NUMBER
locComputeType STRING
statistics STRUCT<
currentServerTime: STRING,
firstLocatedTime: STRING,
lastLocatedTime: STRING

historyLogReason NULL
geoCoordinate NULL
rawLocation NULL
networkStatus STRING
changedOn INTEGER
ipAddress ARRAY
userName STRING
ssId STRING
sourceTimestamp STRING
band STRING
apMacAddress STRING
dot11Status STRING
manufacturer STRING
areaGlobalIdList NULL
detectingControllers STRING
bytesSent INTEGER
bytesReceived INTEGER
guestUser BOOLEAN
)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe';`

[DonDebonair]

Do you have some example JSON data for me that can reproduce this?

[b-rockx]

I believe the following will give you an error. I fixed it manually by using the following code.

def _update_values(self, d):
    for k in d:
        if isinstance(d[k], dict):
            self._update_values(d[k])
        elif d[k] == 'integer' or d[k] == 'number':
            d[k] = 'bigint'
        elif d[k] == 'null':
            d[k] = 'string'

I am currently trying to fix another error that arrises.

if attributes['items']['type'] == 'object':
KeyError: 'type'

I believe this is because 'required' is a key

{
'macAddress':'AA:AA:AA:AA:AA:AA',
'mapInfo':{
'mapHierarchyString':'Market',
'mapHierarchyDetails':{
'campus':'Market ',
'building':'000',
'floor':'001',
'floorAesUid':2349083034,
'zones':''
}
},
'mapCoordinate':{
'x':300.90399,
'y':100.97313,
'z':0.0,
'unit':'FEET'
},
'currentlyTracked':True,
'confidenceFactor':248.0,
'locComputeType':'RSSI',
'statistics':{
'currentServerTime':'2018-05-20T10:24:40.707-0500',
'firstLocatedTime':'2018-05-20T09:33:24.283-0500',
'lastLocatedTime':'2018-05-20T09:33:24.283-0500'
},
'historyLogReason':'MOVEMENT',
'geoCoordinate':None,
'rawLocation':None,
'networkStatus':'ACTIVE',
'changedOn':1526826804283,
'ipAddress':[

],
'userName':'',
'ssId':'',
'sourceTimestamp':'1526826804283',
'band':'UNKNOWN',
'apMacAddress':'',
'dot11Status':'UNKNOWN',
'manufacturer':'Motorola Mobility LLC, a Lenovo Company',
'areaGlobalIdList':None,
'detectingControllers':'10.30.5.5',
'bytesSent':0,
'bytesReceived':0,
'guestUser':False
}