-
Notifications
You must be signed in to change notification settings - Fork 169
Closed
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the bug
Some entities of spark uniffle ui cannot be deserialized in spark history server hybridStore.
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "total" (class org.apache.uniffle.common.ShuffleReadTimes), not marked as ignorable (7 known properties: "backgroundDecompress", "decompress", "crc", "copy", "fetch", "backgroundFetch", "deserialize"])
at [Source: (GZIPInputStream); line: 1, column: 123] (through reference chain: org.apache.spark.AggregatedShuffleReadTimesUIData["times"]->org.apache.uniffle.common.ShuffleReadTimes["total"])
at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:1138)
at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:2224)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1709)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1687)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:320)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4825)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3809)
at org.apache.spark.util.kvstore.KVStoreSerializer.deserialize(KVStoreSerializer.java:70)
at org.apache.spark.status.protobuf.KVStoreProtobufSerializer.deserialize(KVStoreProtobufSerializer.scala:38)
at org.apache.spark.util.kvstore.RocksDB.get(RocksDB.java:169)
at org.apache.spark.util.kvstore.RocksDB.read(RocksDB.java:181)
at org.apache.spark.UniffleStatusStore.shuffleReadTimes(UniffleStatusStore.scala:66)
at org.apache.spark.ui.ShufflePage.doRender(ShufflePage.scala:165)
at org.apache.spark.ui.ShufflePage.render(ShufflePage.scala:473)
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.apache.spark.ShuffleType` (no Creators, like default constructor, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
at [Source: (GZIPInputStream); line: 1, column: 16] (through reference chain: org.apache.spark.ShuffleTaskSummary["shuffleType"])
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1915)
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:414)
at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1360)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserialize(AbstractDeserializer.java:274)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4825)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3809)
at org.apache.spark.util.kvstore.KVStoreSerializer.deserialize(KVStoreSerializer.java:70)
at org.apache.spark.status.protobuf.KVStoreProtobufSerializer.deserialize(KVStoreProtobufSerializer.scala:38)
at org.apache.spark.util.kvstore.RocksDB.get(RocksDB.java:169)
at org.apache.spark.util.kvstore.RocksDB.read(RocksDB.java:181)
Affects Version(s)
0.10.0
Uniffle Server Log Output
Uniffle Engine Log Output
Uniffle Server Configurations
Uniffle Engine Configurations
Additional context
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Metadata
Metadata
Assignees
Labels
No labels