Added multiple DB connection capability

[fozzmoo]

Conflicts:
FS/FS/UID.pm

[freeside]

The intent here is to change the secrets file to support multiple database connection handles

Hi!

This is similar to functionality we removed as part of the move to more modern authentication (commit 9d35792).

So that readonly slaves can be incorporated.

If a readonly slave should be used, a dbh("readonly_slave") can
be called before the readonly work to switch to that DBH. Then, when that is done, olddbh() is called to restore
the original (read-write) DBH to $FS::UID::dbh.

What kind of "readonly work"? Where did you envision it being used or need it?

Interesting for sure and would like to see what you do with it.

Some sort of way to have parts of the web interface pull from read-only slaves? (search/ and report/?) Something in particular slow elsewhere in the interface or API? Some other application?

With regard to the code, probably better to have freeside-upgrade convert the old secrets file format to the new format, rather than carrying legacy code around to read both. (Makefile would need to be updated for the new format if we haven't done something less crufty on the install at that point).

ivan

[fozzmoo]

On Fri, 01 Nov 2013 02:08:44 -0700
"Freeside Internet Services, Inc." notifications@github.com wrote:

= So that readonly slaves can be incorporated. =

If a readonly slave should be used, a dbh("readonly_slave") can
be called before the readonly work to switch to that DBH. Then, when that
is done, olddbh() is called to restore the original (read-write) DBH to
$FS::UID::dbh.

What kind of "readonly work"? Where did you envision it being used or need
it?

Interesting for sure and would like to see what you do with it.

Some sort of way to have parts of the web interface pull from read-only
slaves? (search/ and report/?) Something in particular slow elsewhere in
the interface or API? Some other application?

At Bluehost, we'll be running in a cluster environment and will want to
leverage that by having reports and other non-write operations use
read-only slaves for best performance.

= Code =

With regard to the code, probably better to have freeside-upgrade convert
the old secrets file format to the new format, rather than carrying legacy
code around to read both. (Makefile would need to be updated for the new
format if we haven't done something less crufty on the install at that
point).

That makes complete sense to me.

Doran L. Barton fozz@iodynamics.com - Linux, Perl, Web, good fun, and more!
"No appointment necessary. We hear you coming."
-- Sign seen outside a muffler shop

[Jayceh]

Yeah, the basic idea is once we have easy access to a handle, make things that are known to be read-only (reports for instance), have a very easy/transparent method of running against that slave.
Probably make something very easy to give a hint to qsearch to try to use a slave if available. This way if a person uses the interface to search for a customer, it can run without touching the master. But a bill-now would run as expected against the master.

[freeside]

At Bluehost, we'll be running in a cluster environment and will want to leverage that by having reports and other
non-write operations use read-only slaves for best performance.

Yeah, the basic idea is once we have easy access to a handle, make things that are known to be read-only
(reports for instance), have a very easy/transparent method of running against that slave.
Probably make something very easy to give a hint to qsearch to try to use a slave if available. This way if a
person uses the interface to search for a customer, it can run without touching the master.

Sounds good. Definitely understand the goals / future use cases you outline.

From a technical point of view, we'd probably want:

• A way to hint at both the directory/file level (search/ and graph/) and individual qsearch level
• We'd need to figure out if "readonly ability" is on or off by default. i.e. do we hint for it or "against" it, how that interacts with "hints" at both the page and qsearch level. Consider if it is more complicated than just just a boolean opt-in/opt-out.
• Some sort of middle layer that uses the hinting to control the database connection as appropriate (i.e. do nothing if there's no slave(s) defined, don't re-connect twice even if there were two hints, etc.).

From a "ready for master" / collaborative / cooperative point of view, I'm thinking a topic branch would be a better fit for now? It doesn't seem very open-source of us to merge just the low-level ability to connect to different databases into master without also including the code to make useful use of it.

But a bill-now would run as expected against the master.

It might be worth pointing out related functionality/hinting with regard to "Bill now" and freeside-queued -- the "-s" and "-n" flags are used to . Not the same as read-only access, but related. Perhaps consider an override so the the billing jobs already hinted "secure" for the -s flags would run completely against the master, even for searches that were otherwise hinted "read-only-able"?

[freeside]

obsoleted by newer try #23