Sunday, February 2, 2014

Python Wrapper for the ESRI File Geodatabase API

I have been developing software around ESRIs products for almost 10 years now.  During these years, I have had quite a few ups and downs, including:
  • the transition to 9.x / 10.x
  • dealing with C++ API changes
  • dealing with C# API changes
  • dealing with Python API changes (geoprocessing vs arcpy...)
  • service packs galore
  • transition from foundation / defense to production mapping
  • and much, much, much more...
When ESRI first announced the release of their File Geodatabase API, I was excited!  This would give me cross-platform access to their latest-and-greatest geospatial container in Linux, OSX and Windows!  I use each of these platforms regularly for desktop, development and data processing purposes (though my desktop at home happens to be a MacBook).  Since the release of the first API, I have used it for multiple tasks and it has become a critical part of my day to day development life.

C++ is mighty fine but tonight I needed to interface with the File Geodatabase API using Python (specifically with 2.6..... Do Not Ask!).  I searched high and low and could not find a Python wrapper for it, so I had a fresh cup of Joe and decided to dive right in.  I made my first attempt using SIP but I decided that SWIG would be the easiest route.  After about an hour of fiddling around, I had an interface and a dirty script that would build it all for me.  This is a snapshot of my dirty build script:

export LD_LIBRARY_PATH=FileGDB_API/lib/
swig -python -IFileGDB_API/include -c++ filegdbapi.i
c++ -c filegdbapi_wrap.cxx -IFileGDB_API/include -I/usr/include/python2.6
c++ -shared filegdbapi_wrap.o -LFileGDB_API/lib -lpython2.6 -lFileGDBAPI -o _filegdbapi.so

If you're reading this article and not familiar with software development, all you need to know is that those couple of lines actually do quite a bit.  In the end, two files of interest are generated; filegdbapi.py and _filegdbapi.so, everything I needed to get started!  At this point, we are ready to fire open the Python interpreter and examine our new module:

Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import filegdbapi
>>> 

No errors, that's a good thing!  Let's see what our module has packed inside.

>>> dirs = dir(filegdbapi)
>>> dirs.sort()
>>> 
>>> for d in dirs:
...     print d
... 
BezierCurve
BezierCurve_swigregister
ByteArray
ByteArray_swigregister
CircularArcCurve
CircularArcCurve_swigregister
ClearErrors
CloseGeodatabase
CreateGeodatabase
Curve
Curve_swigregister
DeleteGeodatabase
EllipticArcCurve
EllipticArcCurve_swigregister
EnumRows
EnumRows_swigregister
EnumSpatialReferenceInfo
EnumSpatialReferenceInfo_swigregister
Envelope
Envelope_swigregister
FieldDef
FieldDef_swigregister
FieldInfo
FieldInfo_swigregister
FindSpatialReferenceByName
FindSpatialReferenceBySRID
Geodatabase
...
shapeNull
shapePoint
shapePointM
shapePointZ
shapePointZM
shapePolygon
shapePolygonM
shapePolygonZ
shapePolygonZM
shapePolyline
shapePolylineZM
shapePolylineM
shapePolylineZ

It looks like we are good to go, so let's do something geospatial, like create a new file geodatabase:

>>> g = filegdbapi.Geodatabase()
>>> ret = filegdbapi.CreateGeodatabase('./test.gdb', g)
>>> ret
0

Because FileGDBCore defines S_OK to be ((fgdbError)0x00000000), the value of 0 means successful! Though for sanity, we will check:

>>> import glob
>>> glob.glob('*.gdb')
['test.gdb']

And double check...

Geodatabase creation via our new wrapper, check!  Note to self: make the Python wrapper a little bit nicer by bringing over the integer constants.  Moving on, can I open the file geodatabase that I just created?

>>> g = filegdbapi.Geodatabase()
>>> ret = filegdbapi.OpenGeodatabase('./test.gdb', g)
>>> ret
0

It looks like we're on a roll, so let's try creating a new table in our geodatabase.  I will use the streets.xml straight out of the 'TableSchema' sample provided in the file geodatabase api distribution.

>>> import filegdbapi 
>>> geodatabase = filegdbapi.Geodatabase()
>>> ret = filegdbapi.OpenGeodatabase('./test.gdb', geodatabase)
>>> ret
0
>>> table_def = open('Streets.xml').read()
>>> table = filegdbapi.Table()
>>> geodatabase.CreateTable(table_def, '', table)
0
>>> filegdbapi.CloseGeodatabase(geodatabase)
0

Isn't it nice when things just, work?  Honda sure knows it.  Now it is time to set up an open source project and share this with the world.  I think I will name it the file-geodatabase-api-python-wrapper, ain't that a mouthful!

3 comments:

Kimo said...

Very cool Joe. The first time I have seen a use for SWIG that I considered using years ago when I first started with Python.

How is the performance for loading raw data?

Chuck said...

Thanks for doing the heavy lifting and sharing the process. What is the performance like for loading data, especially bulk loads?

Paulo Pires said...

Hello Joe, is possible on windows?

Thanks