[Prophet] Some random thoughts and questions on OpenResty for Prophet

[agentzh]

Hi, prophet folks!

I've been looking at Prophet and playing with SD for a while. I'm
trying to get my head around it. I know it's still under active
development and lacks documentation (severely). I just want to write
down some things that I'd like to do with Prophet here in the hope of
getting some guidance from the developers.

As one of the authors of OpenResty, I'd really like to hack up an
OpenResty frontend for Prophet so as to make Prophet a database
backend. It'll be great to be able to use our shiny admin site (
http://openresty.org/admin/) to manage the local Prophet replicas as
in ~/sd-bugs/. We can also look forward to more and more shiny web
apps atop Prophet like a p2p BBS ;)

OpenResty's Model API should be relatively easy. It seems the
Prophet::Record class is the right way to go, but I'm still trying to
grok all the API needed to achieve my goal. For the View API, it seems
that we need a SQL-like query language for Prophet (just like GQL for
BigTable and FQL for Facebook). Other APIs like Feed and Action API
are trivial if we get models and views working.

Another interesting thing we've been thinking about is to use Prophet
to serve as an (optional) auxiliary backend storage for the current
PostgreSQL backend in OpenResty. Data synchronization between  Pg and
Prophet themselves deserves more thought. For me, I want Pg only to
store the HEAD snapshot of the Prophet record base, so that we can
enjoy cheap FTI (via tsearch2) and cheap table join and many other
relational goodies. For big data sets that Prophet can't handle very
well, the Prophet support could be disabled via the OpenResty API by
the user.

Well, yeah, I know it's also possible to make Pg a backend for
Prophet. That's just another option, but (IMHO) makes very little
sense to OpenResty itself though.

Help and comments welcome!

-agentzh