On models in a Zend Framework application
Let’s talk about writing models that communicate with databases within a Zend Framework application. It’s a popular topic at the moment as there’s been a few threads recently on the ZF mailing lists about creating models.
The “is a” relationship
When working with models within Zend Framework, the simplest solution is to extend Zend_Db_Table_Abstract and Zend_Db_Table_Row_Abstract along these lines:
class Users extends Zend_Db_Table_Abstract
{
protected $_name = 'users';
protected $_rowClass = 'User';
public function fetchAllInLastNameOrder()
{
return $this->fetchAll(null, array('last_name', 'first_name'));
}
public function fetchUserById($id)
{
$id = (int)$id;
return $this->_fetchRow('id = '. $id);
}
}
class User extends Zend_Db_Table_Row_Abstract
{
public function name()
{
return $this->first_name . ' ' . $this->last_name;
}
}
This works fairly well for models where there is a one to one mapping to the underlying database table. It starts to falls down if a given model needs to know about two or more database tables to do its work.
The knack is to avoid using the functions defined in Zend_Db_Table_Abstract within your models or views as this will lead towards fat controllers that are too aware of the underlying database. This gets quite hard as it’s much easier to start putting model logic into the controllers with this system, especially when it comes to choosing if you need to insert or update a record. A sure sign that you have this problem is that your controllers create SQL statements or Zend_Db_Table_Select objects.
To try and avoid these problems, there have been discussions on the ZF mailing lists for a little while now about using a “has a” relationship from the model object to the Zend_Db_Table object. Bill Karwin has long been a proponent of this idea and recently there’s been much discussion about it. The question is: how do you implement it in practice without ending up writing lots of code for the sake of “cleanliness”?
I’ve started a new ZF project recently where I’ve started playing with these ideas and this is my first attempt at it, so I thought I’d share where I had got to in my thinking.
The “has a” relationship
When implementing this relationship we are trying to hide the fact that we are using Zend_Db_Table from the consumers of the models. The basic idea is that we have a gateway class that holds a protected $_table variable that is an instance of a Zend_Db_Table object. When fetching data, the gateway object will return either a single object or an array of objects of a type which I’ve not-very-imaginatively named “StandardObject“.
This is a picture of the various objects and how they relate to each other:
To get this working, I first mapped out how I want the objects to work via a unit test and then built the classes behind it. I wouldn’t call it TDD though as I didn’t write tests for all the internals as I went along. This is the basic API that I want my consumers to use:
$userGateway = new UserGateway();
// insert a row
$data = array();
$data['first_name'] = 'John';
$data['last_name'] = 'Smith';
$row = $userGateway->setFromArray($data);
// update a row
$data = array();
$data['id'] = 1;
$data['first_name'] = 'Rob';
$data['last_name'] = 'Allen';
$row = $userGateway->setFromArray($data);
// fetch a single user
$row = $userGateway->fetchUserById(1);
// fetch all users
$rows = $userGateway->fetchAllInLastNameOrder();
// delete a user
$result = $userGateway->deleteById(1);
The general idea is that I can write specific functionality into the UserGateway class as I need it. Note also that the row objects returned from the gateway class methods behave a little like ValueObjects only. As they are not attached to the database though, they cannot “save” themselves. (I couldn’t find the “correct” design pattern name, which probably means, that my design is wrong!) This means that all interaction with the database is via the gateway class which makes this design similar in some ways to the Table Module design pattern.
The implementation is fairly involved, so get ready for lots of code!
Firstly we need the Gateway class:
class UserGateway extends App_Model_Db_StandardGateway
{
protected $_tableName = 'UsersTable';
protected $_rowClass = 'UserObject';
public function fetchUserById($id)
{
$id = (int)$id;
return $this->_fetchRow('id = '. $id);
}
public function fetchAllInLastNameOrder()
{
$orderBy=array('last_name', 'first_name');
return $this->fetchAll($orderBy);
}
}
The UserGateway contains the specific methods for this model and has a couple of protected variables that are used to set up the system. We also need a table class, UsersTable that inherits from App_Db_Table_Standard which inherits from Zend_Db_Table_Abstract.
class UsersTable extends App_Db_Table_Standard
{
protected $_name = 'users';
}
Obviously we are going to have many gateway models, so it makes sense that common functionality will be in a parent class, which I’ve called App_Model_Db_StandardGateway. This is the big class and we’ll look at it in chunks:
class App_Model_Db_StandardGateway
{
/**
* @var App_Db_Table_Abstract
*/
protected $_table = null;
protected $_tableName = null;
protected $_rowClass = 'App_Model_StandardObject';
public function __construct()
{
if (empty($this->_tableName)) {
throw new Exception('You must specificy _tableName');
}
$this->_table = new $this->_tableName();
}
Every gateway has a $_table property which is instantiated in the constructor. The name of the table class to instantiate is stored in the $_tableName property which is set in the child class. Finally, we have a $_rowClass property which will be used to hold each row object when the data is retrieved from the database. Note that it defaults to App_Model_StandardObject, but I assume that the child class will override with a specific row class as the UserGateway class does when it specifies that it wants to use the UserObject row class.
The first thing in our API is the ability to create or update database rows. I’ve decided on a function called setFromArray() as a foundational function to allow this. The base implementation is performs these steps:
- Get hold of an instance of the row class
- Call the row’s setFromArray() method
- Call the row’s save() method
This is implemented as:
// (part of App_Model_Db_StandardGateway)
public function setFromArray($data)
{
$row = $this->_fetchRowOrCreate($data);
$row->setFromArray($data);
$row->save();
return new $this->_rowClass($row->toArray());
}
The tricky bit is getting the instance of the row class (an instance of Zend_Db_Table_Row) as we need to either create a new one or retrieve it from the database. Creating a new one is easy: $row = $table->createRow(). However, retrieving a row based on the $data array is more complicated. The simplest way I could think of was to retrieve the primary key info from the table and then call find() with the correct fields from $data array. To do this, I had to extend Zend_Db_Table_Abstract to implement a getPrimary() method like this:
class App_Db_Table_Standard extends Zend_Db_Table_Abstract
{
protected $_rowClass = 'App_Db_Table_Row_Standard';
public function getPrimary()
{
return $this->_primary;
}
}
Note that we also define a custom row class, App_Db_Table_Row_Standard which will allow us to perform operations on insert and delete.
Back to the App_Model_Db_StandardGateway class, we can now create _fetchRowOrCreate():
// (part of App_Model_Db_StandardGateway)
/**
* Either retrieve a row from the DB or create a new one
*
* @param array $data
* @return App_Db_Table_Row_Standard
*/
protected function _fetchRowOrCreate($data)
{
$pk = $this->_table->getPrimary();
if (!is_array($pk)) {
$pk = (array) $pk;
}
$pkData = array();
foreach ($pk as $fieldName) {
if (array_key_exists($fieldName, $data)) {
$pkData[] = $data[$fieldName];
} elseif (!empty($pkData)) {
throw new Exception("Could not find Primary Key part '$fieldName'");
}
}
if (empty($pkData)) {
$row = $this->_table->createRow();
} else {
$rowset = call_user_func_array(array($this->_table , 'find'), $pkData);
if ($rowset) {
$row = $rowset->current();
} else {
$row = $this->_table->createRow();
}
}
return $row;
}
I’m nearly sure this can be improved, but it works for both a simple autoincrement primary key like id and also for compound primary keys of multiple columns which is nice. The method makes an assumption that if you have a compound key, then the $data array will have an entry for all fields that make up the primary key. If not, an exception is thrown. We also assume that if none of the fields that make up the primary key are present in $data, then you want to create a new record.
That covers creating and updating database records from our gateway class. Deletion is a simple proxy directly to the table class:
// (part of App_Model_Db_StandardGateway)
public function deleteById($id)
{
$count = $this->_table->delete('id = '. (int)$id);
return (bool) $count;
}
so let’s look at fetching data. We need two methods in the App_Model_Db_StandardGateway class: _fetchRow() and fetchAll(). These are trivial to implement as we proxy directly to the table class and then all we need to do it turn each row returned into an instance of our StandardObject class:
// (part of App_Model_Db_StandardGateway)
protected function _fetchRow($where, $order = null)
{
$row = $this->_table->fetchRow($where, $order);
if ($row) {
$row = new $this->_rowClass($row->toArray());
}
return $row;
}
public function fetchAll($orderBy=null)
{
$rows = null;
if(is_null($orderBy)) {
throw new Exception('You must choose the order of the results');
}
$rowset = $this->_table->fetchAll(null, $orderBy);
if($rowset) {
$rows = array();
foreach($rowset as $row) {
$rows[] = new $this->_rowClass($row->toArray());
}
}
return $rows;
}
The StandardObject class’ full name is App_Model_Db_StandardObject and is trivial:
class App_Model_Db_StandardObject
{
function __construct($data = array()) {
foreach($data as $fieldName=>$value) {
$this->$fieldName = $value;
}
}
}
Obviously, the intention is that the model specific version, UserObject in this case, is able to do provide model specific functionality. For testing, I simply added a name() method:
class UserObject extends App_Model_Db_StandardObject
{
public function name()
{
return $this->first_name . ' ' . $this->last_name;
}
}
Clearly, I could provide additional methods as required and could overload the constructor if I wanted to do a more complicated mapping from the database row array to the value object representation.
Summary
This entry has turned out much longer than I expected! This is clearly my first attempt at putting the database inside the model and hiding it from the rest of the application. Having implemented this far, I’m not sure if using Zend_Db_Table is actually useful, so my next step will be to remove it and use SQL and a Zend_Db_Adapter instead. If it turns out well, I may blog about it :)
I’m also hoping to spark more debate on the actual implementation of how to do stuff like this. I know Keith Pope is working along similar lines for the example app in his upcoming book, so if you are interested in this approach, have a look at his tactics too as they may make more sense. Matthew Weier O’Phinney has also been doing something similar in his pastebin app which is worth looking at too, though he’s halfway through refactoring.
Obviously, comments and suggested improvements are welcome!
I'll be watching this to see how things crop up. It's a great idea, and something I've pondered for the last little while as some of my apps get bigger.
Hiding all DB functions from the main application and having a gateway control it all also allows you to have a more diverse database back-end (multiple pools of clusters) and inevitably gives you far more scalability.
Yes I think these problems are going to keep coming up and I hope this debate keeps going. However I still agree that the framework should not provide a standard "Model", I think it would cause more problems than it solved. Though I do think there could be more tools available to help people implement the various patterns centered around domain modeling. Again how we go about providing such tools is a big question, though my thinking is that it could be as simple as having a modeling section in the documentation, which would provide detail on the issues and various patterns that can be used to implement the type of model you require, these could almost be ZF Design patterns.
Rob your design looks great, the only comment I have is that once you do:
$row = new $this->_rowClass($row->toArray());
To populate your objects, they have no access to the DB, so you could not get dependent rows etc from your objects. Though you could simply pass in a reference to the row:
$row = new $this->_rowClass($row->toArray(), $row);
But then you will have the problem that the row returns a rowset of Zend_DB_Table_Row and not an object. Hmm see theres lots to think about hehe. You would almost need to then access another gateway that can create the dependent objects.
Hopefully when I get more time I will blog about my original design where I was trying to create domain objects in a similar way, I need to meet my deadline first hehe Oh the joys of writing!
Anyway I hope my ramblings help, and congrats on the book I will be certainly getting a copy.
Keith,
Yes, I've been thinking of modifiying the "StandardObject" so that it has an instance of the row object and then proxy to it with __get() for the properties.
Regards,
Rob…
It turns out that it's quite easy to make the StandardObject act as a gateway to the Zend_Db_Table_Row object.
We just have to rework App_Model_Db_StandardObject so that we pass in the row object into the constructor. We also have to add some proxy functions through to the underlying object:
class App_Model_Db_StandardObject
{
/**
*
* @var App_Db_Table_Row_Standard
*/
protected $_row = null;
public function __get($columnName)
{
return $this->_row->__get($columnName);
}
public function __isset($columnName)
{
return $this->_row->__isset($columnName);
}
public function __set($columnName, $value)
{
return $this->_row->__set($columnName, $value);
}
function __construct(M5_Db_Table_Row_Standard $row) {
return $this->_row = $row;
}
}
Having done this, all calls that create the object are changed from
new $this->_rowClass($row->toArray());
to
new $this->_rowClass($row);
That's all that's needed and now we can add functionality to our StandardObject to allow them to collect dependent rows etc.
Rob…
It's looking good, so fetching dependent object would go something like:
class MyDomainObject extends App_Model_Db_StandardObject
{
public function getSomeDependentObjects()
{
$rows = $this->_row->findDependent('....');
foreach ($rows as $row) {
$this->_items[] = new OtherDomainObject($row);
}
}
}
One other thing you may want to add getter methods for table and tableName, this way you can format the tableName etc, in the App_Model_Db_StandardGateway. Also would it be better as an abstract class?
Also maybe adding setters for table or tableName so that you can add mock tables to the gateway for testing without the db.
Hi rob,
I have been thinking about this all night! So I had a look through the POEAA book this morning :)
Browsing the patterns I was thinking that your gateway actually fits well with the Data Mapper Pattern http://martinfowler.com/eaaCatalog/dataMapper.html
Also with being able to map DB to domain objects means that you can then use repositories to access your domain objects.
However I did also notice another problem with mapping to objects, as soon as you do this you need to be able to track them and more importantly the modifications made to them.
An example maybe if I get a user object and get the users avatar which is a avatar object, if I then modify the avatar via the user object, I then need to be able to track that modification as when I save the User object I would naturally expect that to be saved as well.
Though there are patterns for dealing with this mainly:
http://martinfowler.com/eaaCatalog/identityMap.html
and
http://martinfowler.com/eaaCatalog/unitOfWork.html
Anyway I hope this helps in some way, I best get back to writing…
I also think this example fits in a DataMapper pattern. Though, when relationships come in it's difficult to work this way and Fowler himself suggests to buy (or download) a DataMapper instead of maintain one.
At this time, the only php working DataMapper that I found is Doctrine, that it's not a DataMapper; that's because Doctrine objects are not database-agnostic but are subclass of Doctrine_Record.
Anyone uses a real data mapper in php?
PPoEAA is on my desk at home, so I'll have a read.
I don't think I want anything too complicated though, so I'm happy if my design falls between the official patterns. As long as it works!
Regards,
Rob…
[quote]
It starts to falls down if a given model needs to know about two or more database tables to do its work.
[/quote]
The example of your model design consists of one table, as far as I understood. Which doesn't explain to me how this model could serve "two or more database tables to do its work". Could you provide a short example for UserPets for instance ?
You could maintain an array of tables and access them via a getter. You may also need to maintain multiple row class name that your tables would populate, however this may not be such a good idea, the multiple tables makes more sense if you are creating complex domain objects.
For a example of multiple tables checkout Matthews GIT account and his example apps. (http://github.com/weierophinney)
In a previous 'life' I used value objects within J2EE stateless session beans for talking to PostgreSQL. It was quite painful when any update was made to a database (so much code to refactor).
Anyway…
a) Should App_Model_Db_StandardGateway be an abstract class?
b) It would be nice if there was API compatibility between e.g. Propel, Doctrine… then at least it would be quite simple to switch from one to another (and perhaps encourage people writing their own APIs to follow the same naming convention). (e.g. $foo->setXXX('blah');
$foo->updateFromArray($array);
$foo->save();
$pets = $foo->getBar()->getPets();
FooPeer::retrieveByPk($pk); FooPeer::doSelect($some_sort_of_criteria)).
that's my 2p anyway; it's probably quite worthless.
David.
Hi,
i tryed the example of the models but it doesn't work :(
I always get an infinite loop in the App_Model_Db_StandardGateway (line $this->_table = new $this->_tableName();
)
Is there a link to download the whole script?
Regrads,
Wolfgang
Nice example, but I still have a concern over the fact that you directly expose values that should probably be hidden / protected. You have moved the actual Zend_Db_Table_Row stuff down into a few depths of code, but some other part of your application could still do something like this:-
$user = $userGateway->fetchUserById(1);
$user->id = 2;
$user->createdOn = '2009-01-01';
$user->save();
Oops! We just overwrote user 2 with user 1's data!
I am going to have to pull my finger out and setup my own blog and get round to reading that fowler book that everyone references so I can give a more detailed outline of my views on this topic – damn clients keep getting in the way though ;-)
Surely, you can't stop something like that as the rest of the application has to be able to insert new records and update current ones.
You can make your "value object" clever enough to prevent changing the id and other 'meta' fields such as created_on though.
Regards,
Rob…
It would be nice, if somebody help me with my problem.
I always get an infinite loop at the line $this->_table = new $this->_tableName() :(
Regards,
Wolfgang
While I am not ready to publish my code, I can tell you I have implemented the Mapper approach as in Fowler, including Identity Maps and the Unit of Work approach.
As a result, mappers are used to load objects, but once loaded they are tracked by IdentityMaps owned by the UOW. At the very end of the page, all I do is commit the UOW and all that needs saving, inserting, or updating gets done.
What is needed, however, is to give the UOW implementation a topological sort of the Model object classes so it knows in which order to save them (this causes id fields in related models to be set upon insert of a newly created model object, etc.).
Very interesting Dolf, I would point you to the data mapper proposal currently being worked on here
http://framework.zend.com/wiki/display/ZFPROP/Zend_Db_Mapper+-+Benjamin+Eberlei?focusedCommentId=9437718
I am sure if you have implemented this the guys would be very happy to have your input.
Hi Rob
Great ideas. I really like your code / model design and would like to use some of your ideas for my next work. As I just started using PHPUnit it would be great if I could get the test code too, to see how you have set it up. Would that be possible?
Thanks for sharing your ideas & work. Nicolas
Hi Nicolas,
Feel free. See http://akrabat.com/2008/06/25/license-for-code-published-on-this-site/
Regards,
Rob…
Hi Rob
Thanks for the information. But what would be great for me is to also see the tests you have written. The code that leaded to the code you published in your article. Would that be possible?
Best regards
Nicolas
The example of your model design consists of one table, as far as I understood. Which doesn't explain to me how this model could serve "two or more database tables to do its work". Could you provide a short example for UserPets for instance ?
Hi,
It might be a pretty dumb question to ask, but why the extra effort to create lots of classes and extending them to hide Zend_db from your application individually.
The problem IMHO, is to hide Zend_Db from your application. And when I mean Zend_Db I talk about everything. Preventing the usage of it's methods, like select or something, somewhere else then your models, because you might need to move on from connecting to databases to a web service to get your data.
So All we need to do is creating methods on my models and only using them to get/save/update data. About more than one tables, if you start writing your own queries in your model, you will write every query you want. The next problem occurs because of usage of Zend_Db_Table's methods like fetch which returns a Zend_Db_Table_Row or equivalent. When you start to write your own sql queries, you can tell to fetch results to Objects, which then creates a php built-in StandardObject.
No need to create lots of extra classes and lines of code to hide all the zend_db layers individually. My approach is basically hiding everything behind one model, which is used everywhere in my application.
I understand that the model here is the next iteration of my approach. With adding extra abilities to the StandartObject and basically improving it's usage (like adding save or update features (powers?) on it, I do this with a save method on my model which takes an associative array and executes an update/insert statement).
The model here might be a better OOP approach too, I just want to figure out why all the extra work?
Sorry if my question is very dumb and noobish.
thanks
Roy,
In my experience, it requires a lot of self-discipline to not do stuff in the controller that should be in the model. With a team of programmers of varying abilities, it's best to make it extremely hard to do it wrong.
Regards,
Rob…
I guess it's a Java-like approach, force developer to do the right thing because he might not always be at a good state to do the right thing :)
I do not completely agree with the idea, because I believe a developer must be an intelligent guy. After reading something like this on Matthew Weier O'Phinney's blog, I guess I understood better of what we are trying to do here. It's not only about hiding Zend_Db, because we are not only talking about one model here. I was thinking like that (there is only user and nothing else), but there is also a list of users and some other things too. So you should be able to hide them too and this is what you are doing. Thanks for the reply.
And about App_Db_Table_Row_Standard? Can you show the code of this class?
Thanks…
Has_a instead of Is_a wherever we can, favor composition over inheritance is what Kathy Sierra taught me. Thanks for this excellent elaboration.
My last big project involved extremely complex objects composed of gobs of tables. It wasn't Zend but it was MVC. I had lotsa code in the models and it turned me into a disbeliever in strict ORM, but there was no abstract factoring of common gateway functionality partly cuz it was in PHP4 and partly cuz I never expected it to get so complicated. So now I have to decide whether I can remember these wrappers long enough to write models on top of them.
While I think I sorta get the idea behind this, it seems that this approach falls short when you need to build more complex queries using joins. All of the examples I have seen deal only with a single table using 1:1 mappings. Can someone please explain how you would go about modifying this approach to use multiple tables with joins?
Sorry if this question seems "noobish".
Thanks in advance!
Matt
It is not banal: you have to resort to an Orm.
I have developed an application to manage and organize knowledge based on topic maps with the Zend Framework: http://www.quesucede.com. The Zend Framework was both very productive and relatively easy to use.
Hi,
I have also been thinking about models in ZF, and for me, gateway is a good idea, while rewrapiping Zend classes into your own is not. Zend Framework has a brilliant system of model containers, with security features as disallowing a modification of primary key, etc, and there's no use in making one's own model containers.
By contrast, there's a lot of use for gateways. First, you separate it from the Zend_Db_Table class, which has a lot of brilliant features, but it makes it quite complex, though, and so it's easy to break sth up wehen adding one's own functionalities. Also, "Gateway" model is a great place to put A LOT of logic related to some entity. For example, for the User entity, you may add functions as
registerAccount()
blockAccount()
unblockAccount()
checkPermissionsFor(module, controller, action)
… etc
Also, if you use few different data sources, as DB, XML and CSV, you may pack them all inside model, ie:
protected $_dbTable for DB
protected $_xmlTable for XML
protected $_csvTable for CSV
Greetings,
Chris