[Prophet] Some random thoughts and questions on OpenResty for Prophet

[agentzh]

On Fri, Oct 17, 2008 at 5:35 PM, Jesse Vincent <jesse at bestpractical.com> wrote:
>
> If you can list off things you want better docced, that can help steer
> us toward improving those things.
>

Hmm, I can name some:

* Replica manipulation API
* Type schema manipulation API
* Record manipulation API
* Record Query API
* Replica reflection API
* Backend (git/svn/etc) configuration

>
> That seems about right. Prophet doesn't require that multiple objects of
> the same type have the exact same schema, so we may have to be careful
> adapting the Model API.
>

*nod*

Prophet is not a true relational database.

What I want to see first though is some kind of RDBMS emulation so
that I can reuse the existing OpenResty test suite and making existing
web apps for OpenResty run as soon as possible :) After that I can
tune the API to expose more flexibility (and power) of Prophet.

> What methods do we need to implement to be able to support your model
> API?
>

Well, model is just a table in the RDBMS sense. OpenResty's Model API
consist of the following:

* Model CRUD  (table schema CRUD)
* Model column CRUD  (table column schema CRUD)
* Model row CRUD  (record CRUD)

>
> Well, we need a query language. "SQL-like" kind of makes me sad because
> SQL is such a usability nightmare for developers, but I
> wouldn't turn down a developer who wanted to work on something like that
> :)
>

That's fine. SQL is by no means the best query language and I often
get annoyed by some lousy SQL queries in $work as well. But it's a
thing and often the only choice in most RDBMS :) I also love to see an
alternative which outperforms SQL in at least the cases where SQL does
not fly.

>
> Implementing alternate Prophet backends that store in a relational
> database is definitely doable, but we'll have to be careful about
> forcing exact schema on individual records.
>

Agreed. I'm personally more interested in a git backend :P

>
> In prophet, we have tools to support sync with a foreign database type,
> but most often it makes sense to customize such things for the foreign
> db's fixed schema.
>

Such things look very tricky to me :)

>
> I'd really like those to be available to Prophet users without needing
> to rely on a Postgres installation. One option here is to adopt some of
> the work Yuval Kogman (nothingmuch) has been doing on KikouDB.
>

Right. Combined Pg and prophet storage is an OpenResty-centric point
of view :P A prophet-backed OpenResty is much easier in the short term
:)

>
> *nod* I think I'm most excited about making Prophet a sane backend for
> OpenResty.  One option MIGHT be to let OpenResty use another "remote"
> REST API as a backend store. That's something we've been considering
> doing in Jifty forever.
>

Yup, that'd be cool if we can invoke foreign REST APIs in OpenResty's
actions. My only concern is performance penalty for personal use :)

> Should we plan a hackathon on some of this stuff around the Beijing Perl
> Workshop?
>

According to qiang++ there is going to be a hackathon after the BPW
and maybe we can look into more details there and hopefully get
something runnable ;)

Thanks!
-agentzh