Refreshing the linked Retrieval Cache data
An important challenge of the DDMS Client is to create a lot of
flexibility for refreshing the local data in accordance with the needs of
the business application. The crux in this is efficiency. If there were no
time and resources involved it would of course be great to completely
refresh all data very often. However, in practice this would cause a
potentially large amount of data to be transferred across the network,
causing delays and high demand on computer's CPUs and network bandwidth.
So the challenge for the DDMS Client is to allow for data fragments to be
strategically refreshed at appropriate moments.
An additional factor that makes this even more challenging is that
the DDMS has been designed to work in the context of any PHP database. In
other words: it does not make assumptions about there for example being a
'date_last_updated' field in linked data tables, based on which a decision
can be made to update data records.
In response to this challenge, the following functionality was
developed to handle the refresh data event.
There are two basic types of refresh events in the DDMS Client: ,
and a. The 'All Data' refresh event
The 'All Data' refresh event, which does exactly what it's name
promises: it replaces all data that is held in the local cache with
data from the DoR.
The 'Refresh Data Fragment' refresh event, which will be
described in more detail below
The 'Refresh Table Fragment' refresh event, which provides the
hosting business application with a powerful way to directly request
for a refresh of a specified horizontal fragment of one of the linked
tables. This functionality has not yet been implemented, and the
functional requirements will be described below.
The Refresh Data Fragment event
The 'Refresh Data Fragment' event works as follows: It is run only
for those linked tables that have the field 'local_refresh' set to have
the value 1 in the linked_tables table. During this event there are
three distinct phases:
Delete: during this phase the records that were deleted from
the DoR are cleared from the local Retrieval Cache.
Update: during this phase all fields in a selected set of
records in the local retrieval cache are replaced with the values in
their DoR countreparts. The way to determine which f records to
update will be described a bit later.
Insert: during this phase all records that were newly inserted
in the DoR since the last refresh data event ran are inserted into
the local retrieval cache.
The Update phase
The rules to determine which records must be updated are encoded
in the UpdateLocalDataRule object that was introduced briefly above.
This object plays an important role in interfacing the DDMS Client
with its hosing business application. Simply explained it works like
this: during the update phase the ExternallyLinkedOrm object will
instantiate an instance of the UpdateLocalDataRule class, call the
newly created object's applyRule() method. This method returns a set
of primary key values to the ExternallyLinkedOrm object, the records
corresponding to which need to updated. The ExternallyLinkedOrm object
then sends this key set to the DDMS Server which in response sends
back the set of records that were requested from it. The DDMS Client
then proceeds the Update Phase by replacing records held in local
retrieval cache with the records it received from the DDMS
Server.
Cascading update, delete, insert
To make the Refresh Data Fragment event more sophisticated each
of its three phases (Delete, Update, Insert) optionally includes
respectively a cascading delete, cascading update, and cascading
insert. The meaning of this will be explained in this section. Be
careful, reading this could result in a headache is you're not in the
right space for it......! ;-)
As you'll see, the relationships between linked table is crucial
in these cascading refresh events. In the description below I'll refer
to the the following
Current Table: table whose data is currently in the process
of being refreshed.
Referenced Table: table that the Current Table has a foreign
key relationship with.
Referencing Table: table that has a foreign key relationship
with the Current Table.
Cascading Delete
Cascading Delete means that for each record that needs to be
deleted in the Current Table, the DDMS Client checks for records in
other linked tables that have a foreign key relationship with the
record that is about to be deleted, and runs the regular Update Phase
for these records.
Cascading Update
As part of the Update Phase, the Cascading Update option means
that for the Current Table, in addition to the set of records that is
updated as defined in the UpdateLocalDataRule object, an additional
set of records is updated, based its Referenced Tables. For each of
its Referenced Tables, first the applyRule() method in the
UpdateLocalDataRule object is invoked, resulting in a set of primary
key values for the Referenced Table. Based on this set of primary key
values, which are foreign key values Current Table, a set of primary
key values is retrieved from the Current Table, and the records
corresponding to it are updated in the way described above.
Cascading Insert
Cascading Insert means that for each record that is newly
inserted into the local retrieval cache as part of the Insert Phase of
the Refresh event, for each Referencing Table the DDMS Client requests
from the DDMS Server the set of Records from the Referencing Table
that refer to the newly inserted record of the Current Table. On the
Client Side the corresponding records in the Referencing table will
then be updated with these records.
Example
Let's lighten up this heavy going stuff with a humorous example
to illustrate the usefulness of these cascading refresh events.
Imagine the purely hypothetical situation that Germany decided it
would be better off if all of its citizens spoke Dutch and if it would
join the Netherlands. Imagine that the UpdateLocalDataRule in France's
instance of a business application results in only student records for
France to be updated during the Refresh Data Fragment event, and we
would not have the cascading refresh events described above. The
result would be that from the Countries table in the database the
record for Germany would be dutifully deleted, but the Student Table
would still have many records that refer to a by now non-existent
Germany record in the Country table
Now imagine the same scenario but this time with the Cascading
Delete event as part of it. Even though initially only all the French
student records will be updated in the DDMS Clients retrieval cache,
when the time comes to delete the Germany record from the Countries
table the system would actually go and refresh every record in the
student table that refers to the Germany record. The result this time
would be that the DDMS Clients set of data would reflect the new
geopolitical situation and all students formerly known as german would
now be dutch in the database.............
Cascading Refresh Config Settings
Back to reality: as was said before, the cascading refresh
events are optionally. This means that for each reference in the
referenced_tables system configuration table the following three
values need to be present
cascading_update
cascading_delete
cascading_insert.
For each of these fields, if their value is 1 this indicates a
cascading update, delete, or insert, if the value is 0, this means the
reference does not cascade its update, delete or insert events.
The Refresh Table Fragment Refresh Event
As said above, there is a need to implement a kind of
functionality that allows the hosting business application to request a
refresh of a horizontal fragment of a specific linked table's data. For
example, if a person logs into the system and the login credentials
cannot be matched with any person's record in the person table, the
system could request a refresh of the person table of all records that
match the username supplied by the user. This is to cater for the
situation that the user's login credentials were updated in the DoR and
this update not having been made in the local Retrieval Cache
yet.
Staging new data records for insertion into the DoR
This refers to newly created data records on the DDMS Client Side
that has the DDMS Server side as its DoR. Functionality on both Client
and Server Side needs to be developed to implement this functionality.
This will be part of the next development phase.
