geomesa hbase为已创建好的schema添加字段

其他 2020-03-19 08:50:22 阅读次数: 0

记录一次为已创建好的schema添加字段，使用hbase作为数据源。
当迭代不断的向前业务不断的加大，导致前期设计的schema无法满足现在业务的需求，需要在已创建好的schema上新增字段以此来满足需求。
schema新增字段很自然的会想到更新schema，刚好api接口中有个接口void updateSchema(String typeName, SimpleFeatureType featureType) throws IOException，于是在代码中创建一个SimpleFeatureType添加新的字段，调用updateSchema函数修改指定的schema。以为就此大功告成，没想到报错，提示

updating schema columns is not allowed

查看该api接口的源码发现不支持修改已经创建好的schema的字段，源码如下图所示： 在这里插入图片描述
提供的api接口不支持修改已创建schema的字段，但是发现在指定的catalog下创建一个schema都会在hbase中该catalog的元数据表中添加如下图所示信息：
在这里插入图片描述
上图中可以看到rowkey为*attributes的value记录着schema的元数据，尝试使用hbase shell的put命令修改该value的值进行新增，put命令为：

put ‘test’,“testtypename~attributes”,“m:v”,“myid:String,value:String:index=true,temp:String,dtg:Date,*geom:Point:srid=4326;geomesa.keywords=‘testkeywords’,geomesa.index.dtg=‘dtg’,geomesa.table.sharing=‘false’,geomesa.indices=‘z3:2:3,z2:2:3,id:1:3,attr:5:3’,geomesa.z3.interval=‘week’”

其中temp:String为新增字段，设置成功，并且新增数据也可以成功，但是查询的时候报

ERROR Invalid TWKB geometry type 0 java.lang.IllegalArgumentException:
Invalid TWKB geometry type 0 at
org.locationtech.geomesa.features.serialization.TwkbSerialization c l a s s . d e s e r i a l i z e ( T w k b S e r i a l i z a t i o n . s c a l a : 198 a t o r g . l o c a t i o n t e c h . g e o m e s a . f e a t u r e s . k r y o . s e r i a l i z a t i o n . K r y o G e o m e t r y S e r i a l i z a t i o n class.deserialize(TwkbSerialization.scala:198 at org.locationtech.geomesa.features.kryo.serialization.KryoGeometrySerialization .deserialize(KryoGeometrySeriali
at
org.locationtech.geomesa.features.kryo.impl.KryoFeatureDeserializationKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲matchReader$8.a…anonfun$matchReader 8. a p p l y ( K r y o F e a t u r e a t o r g . l o c a t i o n t e c h . g e o m e s a . f e a t u r e s . k r y o . i m p l . A c t i v e D e s e r i a l i z a t i o n 8.apply(KryoFeature at org.locationtech.geomesa.features.kryo.impl.ActiveDeserialization class.readFeature(ActiveDeserialization.scala
at
org.locationtech.geomesa.features.kryo.impl.ActiveDeserialization c l a s s . d e s e r i a l i z e ( A c t i v e D e s e r i a l i z a t i o n . s c a l a a t o r g . l o c a t i o n t e c h . g e o m e s a . f e a t u r e s . k r y o . K r y o F e a t u r e S e r i a l i z e r class.deserialize(ActiveDeserialization.scala at org.locationtech.geomesa.features.kryo.KryoFeatureSerializer MutableActiveSerializer.deserialize(KryoFeatureSer
at
org.locationtech.geomesa.hbase.index.HBaseIndexAdapterKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲org$locationtec…anonfun o r g org locationtech g e o m e s a geomesa hbase i n d e x index HBaseIndex
at scala.collection.IteratorKaTeX parse error: Can't use function '$' in math mode at position 5: anon$̲11.next(Iterato…anon 2. n e x t ( C l o s e a b l e I t e r a t o r . s c a l a : 36 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . c o l l e c t i o n . S e l f C l o s i n g I t e r a t o r 2.next(CloseableIterator.scala:36) at org.locationtech.geomesa.utils.collection.SelfClosingIterator $anon 1. n e x t ( C l o s e a b l e I t e r a t o r . s c a l a : 162 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . c o l l e c t i o n . C l o s e a b l e I t e r a t o r 1.next(CloseableIterator.scala:162) at org.locationtech.geomesa.utils.collection.CloseableIterator $anon 5. n e x t ( C l o s e a b l e I t e r a t o r . s c a l a : 101 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . c o l l e c t i o n . C l o s e a b l e I t e r a t o r 5.next(CloseableIterator.scala:101) at org.locationtech.geomesa.utils.collection.CloseableIterator $anon 5. n e x t ( C l o s e a b l e I t e r a t o r . s c a l a : 101 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . i n d e x . g e o t o o l s . G e o M e s a F e a t u r e R e a d e r W i t h A u d i t 5.next(CloseableIterator.scala:101) at org.locationtech.geomesa.index.geotools.GeoMesaFeatureReaderWithAudit a n o n f u n anonfun next 2. a p p l y ( G e o M e s a F e a t u r e R e a d e a t o r g . l o c a t i o n t e c h . g e o m e s a . i n d e x . g e o t o o l s . G e o M e s a F e a t u r e R e a d e r W i t h A u d i t 2.apply(GeoMesaFeatureReade at org.locationtech.geomesa.index.geotools.GeoMesaFeatureReaderWithAudit a n o n f u n anonfun next 2. a p p l y ( G e o M e s a F e a t u r e R e a d e a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . s t a t s . M e t h o d P r o f i l i n g 2.apply(GeoMesaFeatureReade at org.locationtech.geomesa.utils.stats.MethodProfiling class.profile(MethodProfiling.scala:19)
at
org.locationtech.geomesa.index.geotools.GeoMesaFeatureReaderWithAudit.profile(GeoMesaFeatureReader.scala:90)
at
org.locationtech.geomesa.index.geotools.GeoMesaFeatureReaderWithAudit.next(GeoMesaFeatureReader.scala:103)
at
org.locationtech.geomesa.index.geotools.GeoMesaFeatureReaderKaTeX parse error: Can't use function '$' in math mode at position 5: anon$̲2.org$locationt…anon 2. n e x t ( G e o M e s a F e a t u r e R e a d e r . s c a l a : 65 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . i n d e x . g e o t o o l s . G e o M e s a F e a t u r e R e a d e r 2.next(GeoMesaFeatureReader.scala:65) at org.locationtech.geomesa.index.geotools.GeoMesaFeatureReader $anon 2. n e x t ( G e o M e s a F e a t u r e R e a d e r . s c a l a : 65 ) a t o r g . g e o t o o l s . f e a t u r e . F e a t u r e R e a d e r I t e r a t o r . n e x t ( F e a t u r e R e a d e r I t e r a t o r . j a v a : 73 ) a t o r g . g e o t o o l s . f e a t u r e . F e a t u r e R e a d e r I t e r a t o r . n e x t ( F e a t u r e R e a d e r I t e r a t o r . j a v a : 43 ) a t o r g . g e o t o o l s . f e a t u r e . c o l l e c t i o n . D e l e g a t e F e a t u r e I t e r a t o r . n e x t ( D e l e g a t e F e a t u r e I t e r a t o r . j a v a : 66 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . c o l l e c t i o n . C l o s e a b l e I t e r a t o r 2.next(GeoMesaFeatureReader.scala:65) at org.geotools.feature.FeatureReaderIterator.next(FeatureReaderIterator.java:73) at org.geotools.feature.FeatureReaderIterator.next(FeatureReaderIterator.java:43) at org.geotools.feature.collection.DelegateFeatureIterator.next(DelegateFeatureIterator.java:66) at org.locationtech.geomesa.utils.collection.CloseableIterator $anon 4. n e x t ( C l o s e a b l e I t e r a t o r . s c a l a : 51 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . c o l l e c t i o n . C l o s e a b l e I t e r a t o r 4.next(CloseableIterator.scala:51) at org.locationtech.geomesa.utils.collection.CloseableIterator $anon 4. n e x t ( C l o s e a b l e I t e r a t o r . s c a l a : 49 ) a t s c a l a . c o l l e c t i o n . I t e r a t o r 4.next(CloseableIterator.scala:49) at scala.collection.Iterator class.foreach(Iterator.scala:742) at
org.locationtech.geomesa.utils.collection.CloseableIteratorKaTeX parse error: Can't use function '$' in math mode at position 5: anon$̲4.foreach(Close…anonfun$export 1. a p p l y ( E x p o r t C o m m a n d . s c a l a : 84 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . t o o l s . e x p o r t . E x p o r t C o m m a n d 1.apply(ExportCommand.scala:84) at org.locationtech.geomesa.tools.export.ExportCommand a n o n f u n anonfun export 1. a p p l y ( E x p o r t C o m m a n d . s c a l a : 84 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . i o . W i t h C l o s e 1.apply(ExportCommand.scala:84) at org.locationtech.geomesa.utils.io.WithClose .apply(SafeClose.scala:51)
at
org.locationtech.geomesa.tools.export.ExportCommand c l a s s . e x p o r t ( E x p o r t C o m m a n d . s c a l a : 84 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . h b a s e . t o o l s . e x p o r t . H B a s e E x p o r t C o m m a n d . e x p o r t ( H B a s e E x p o r t C o m m a n d . s c a l a : 19 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . t o o l s . e x p o r t . E x p o r t C o m m a n d class.export(ExportCommand.scala:84) at org.locationtech.geomesa.hbase.tools.export.HBaseExportCommand.export(HBaseExportCommand.scala:19) at org.locationtech.geomesa.tools.export.ExportCommand class.export(ExportCommand.scala:77)
at
org.locationtech.geomesa.hbase.tools.export.HBaseExportCommand.export(HBaseExportCommand.scala:19)
at
org.locationtech.geomesa.tools.export.ExportCommandKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲execute$2anonfun$apply 1. a p p l y ( E x p o r t C o m m a n d . s c a a t o r g . l o c a t i o n t e c h . g e o m e s a . t o o l s . e x p o r t . E x p o r t C o m m a n d 1.apply(ExportCommand.sca at org.locationtech.geomesa.tools.export.ExportCommand a n o n f u n anonfun execute 2 2 a n o n f u n anonfun apply 1. a p p l y ( E x p o r t C o m m a n d . s c a a t o r g . l o c a t i o n t e c h . g e o m e s a . t o o l s . D a t a S t o r e C o m m a n d 1.apply(ExportCommand.sca at org.locationtech.geomesa.tools.DataStoreCommand class.withDataStore(Command.scala:60)
at
org.locationtech.geomesa.hbase.tools.export.HBaseExportCommand.withDataStore(HBaseExportCommand.scala:19)
at
org.locationtech.geomesa.tools.export.ExportCommandKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲execute$2.apply…anonfun$execute 2. a p p l y ( E x p o r t C o m m a n d . s c a l a : 45 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . u t i l s . s t a t s . M e t h o d P r o f i l i n g 2.apply(ExportCommand.scala:45) at org.locationtech.geomesa.utils.stats.MethodProfiling class.profile(MethodProfiling.scala:26)
at
org.locationtech.geomesa.hbase.tools.export.HBaseExportCommand.profile(HBaseExportCommand.scala:19)
at
org.locationtech.geomesa.tools.export.ExportCommand c l a s s . e x e c u t e ( E x p o r t C o m m a n d . s c a l a : 45 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . h b a s e . t o o l s . e x p o r t . H B a s e E x p o r t C o m m a n d . e x e c u t e ( H B a s e E x p o r t C o m m a n d . s c a l a : 19 ) a t o r g . l o c a t i o n t e c h . g e o m e s a . t o o l s . R u n n e r class.execute(ExportCommand.scala:45) at org.locationtech.geomesa.hbase.tools.export.HBaseExportCommand.execute(HBaseExportCommand.scala:19) at org.locationtech.geomesa.tools.Runner class.main(Runner.scala:28)
at
org.locationtech.geomesa.hbase.tools.HBaseRunner$.main(HBaseRunner.scala:21)
at
org.locationtech.geomesa.hbase.tools.HBaseRunner.main(HBaseRunner.scala)

看错误好像是序列化错误，想到是不是因为添加的字段在已存在的字段之间导致查询到的未添加新字段之前的数据无法序列化为已添加新字段的结构中。于是尝试将新增的字段放到最后，重新修改hbase中的元数据表，修改后如下图所示：
在这里插入图片描述
其中add:String为新增字段。修改完之后进行插入数据和查询数据都能成功，查询到的数据中未添加新字段前的数据在新字段这一列为空。
至此为已创建的schema新增字段完成，本次是使用hbase作为数据源，其他的数据源应该也可以通过修改元数据的方式进行新增字段。

Demo_zfs
发布了11 篇原创文章 · 获赞 3 · 访问量 601
私信 关注

猜你喜欢

转载自blog.csdn.net/zfs_sir/article/details/87881778
今日推荐
《美国对全球网络空间安全与发展的威胁和破坏》报告发布
火速冲上 GitHub 热榜 —— 开源编程语言、框架哪有这么可爱？
北京人形机器人创新中心发布全球首个纯电驱拟人奔跑的全尺寸人形机器人“天工”
LFOSSA 源来如此公开课 | 掌握云原生未来：CNCF 认证全面攻略与备考秘籍
国产云输入法——仅华为无云端数据上传安全问题
周排行
Python环境安装与基础语法（1）——计算机基础知识
IMU预积分
ADAS中的LDW、FCW、BSD、LCA、ACC、AEB、APA、DMS代表的含义
B站笔试两道题
skyeye arm 硬件虚拟机环境的搭建
Web前端静态页面示例
数组-合并排序数组 II-简单
springcloud之版本问题启动报错
面向对象-------------匿名对象(六)
输入URL到页面呈现中间发生了什么？
每日归档
更多
2024-04-30(1)
2024-04-29(40)
2024-04-28(0)
2024-04-27(56)
2024-04-26(39)
2024-04-25(22)
2024-04-24(36)
2024-04-23(26)
2024-04-22(39)
2024-04-21(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022