I notice that in charge , counter in Zope2 can generate some conflict error.

Why ?

Because two thread want to change the value of a variable. Conflict error are exposed here :
http://wiki.zope.org/zope2/ConflictErrors

But in certain case it’s useful to have an global counter that increment in certain operation. Cache Fu have those counter for caching purpose. Or in charge those counter generate some Conflict Error. There is an solution : resolve the conflict by hand.

In zope source I notice that there is an class that implement this use case. I try to add an counter with this implementation. And I test two implementation of the two counter : one wich was simply an int and one wich was Products.Transience.Transience.Increaser.

The init code of the first counter looks like that :

tool.counter = Products.Transience.Transience.Increaser(0)

The second counter looks like that:

tool.counter2 = 0

I test under siege the incrementation of the two implementation:

First test (Increaser)

Transactions:		         200 hits
Availability:		      100.00 %
Elapsed time:		        6.68 secs
Data transferred:	        0.00 MB
Response time:		        0.10 secs
Transaction rate:	       29.94 trans/sec
Throughput:		        0.00 MB/sec
Concurrency:		        2.97
Successful transactions:         200
Failed transactions:	           0
Longest transaction:	        0.44
Shortest transaction:	        0.01

O conflicts errors

Second test (int)

## 20 user

Transactions:		         200 hits
Availability:		      100.00 %
Elapsed time:		       11.35 secs
Data transferred:	        0.06 MB
Response time:		        0.26 secs
Transaction rate:	       17.62 trans/sec
Throughput:		        0.00 MB/sec
Concurrency:		        4.56
Successful transactions:         197
Failed transactions:	           0
Longest transaction:	        6.09
Shortest transaction:	        0.01

49 conflicts (3 unresolved)

I just release a little tool to detect Memory Leak in zope2 call Products.MemoryProfiler .

It use heapy (http://guppy-pe.sourceforge.net/#Heapy) in internal. It’s just an interface to this tool.

It provide an http interface in zope control panel to see the current memory .

When you start profiling, you take an snapshot of the memory at instant t.
When you click to updateSnapshot, memory profiling  tell you what objects are added between the start and the updateSnashot click. It will be usefull to detect Memory Leak.
Each snapshot is store (as string) in MemoryProfiler to be consult later (link to the date).

The button clear db cache clear all zeo cache of all mounting point so you can see the impact of the memory of those cache.

For windows users, you must compile guppy. There is egg for python 2.6 but
no for python 2.4. I have fatal error with Mingw to compile guppy. I hope that we have soon a binary egg to for python 2.4.

I hope that this tool give to us usefull  information to the memory consume by zope.

You want to make an authenticated cache with apache/squid-varnish/plone and you don’t
know how do that : it’s possible with the vary tag.

Vary header tell to proxy cache what’s headers is variant for an object for a cache.
For example if you tell to the cache that the variant is Cookie , then for a same url with different cookie value the result of the cache is different.
The Server send to the proxy (in the response) which header is considered for vary by sending Vary: list of request header name

In Cachefu, you can configure that by rule with varyExpression.

In global configuration of cache fu you can also configure an global vary header. By default this configuration is send with rule.portal_cache_settings.getVaryHeader()

You can activate or desactivate vary with the header_set configuration ( vary field).

Vary headers must be present in the request (not response) of the browser in order to be considered to be variant for the proxy cache. So we are limited with the standard header of the protocol http.

But with cookie and apache (apache is in front of squid) we can elaborate strategy to construct a vary tag more efficient.

The second aspect of the cache work is purge content when the content change.

PURGE of Vary objects is still very poorly supported in squid, and you can only purge one variant at a time and need to get the URL cached again before being able to purge another variant. So how to deal with that also ?

First , how build our vary tag ?

The trick is to construct an custom vary tag with apache.
We can to do this with RewriteRule::

RewriteCond %{HTTP_COOKIE} mycookie="([^"]+) [NC]
RewriteRule ^(.*)$ - [E=mycookie:%1]

So in this example mycookie contains the value of cookie_key

You can add a cookie for the language , a cookie for group , a cookie for a permission and so on and then construct your custom vary tag with values of this specifics cookies with mod_headers

RequestHeader append MyVary %{mycookie}e

And then the value of mycookie is considered to be variant..

If you want have a specific vary tag for anonymous you can test the presence of
__ac cookie and send a custom MyVary in this case

RewriteCond %{HTTP_COOKIE} __ac="([^"]+) [NC]
RewriteRule ^(.*)$ - [E=authenticated:1]

RequestHeader append MyVary %{mycookie}e env=authenticated
RequestHeader append MyVary anonymous env=!authenticated

So now with that you can vary cache as you want. Now how to treat the big deal of purge.

The trick is  have an image (or a ajax request or ..) in content that is never in cache. This image is serve by a browser view (in case of zope application) that set a cookie. This cookie value is added to Vary tag. So the Vary tag change if the value of this cookie change and then the content is updated (for all request).

For example we can construct a cookie with the value of the catalog change

catalog_count = pcs.getCatalogCount()
context.REQUEST.RESPONSE.appendHeader('Pragma','no-cache')
context.REQUEST.RESPONSE.appendHeader('Cache-control', 'no-cache')
cookie = context.REQUEST.cookies.get('X_CACHE_CATALOG', 0)

if cookie != str(catalog_count) :
context.REQUEST.RESPONSE.setCookie('X_CACHE_CATALOG',
catalog_count ,
path="/")
return catalog_count

And in apache we add
RewriteCond %{HTTP_COOKIE} X_CACHE_CATALOG=([^"]+) [NC]
RewriteRule ^(.*)$ - [E=X_CACHE_CATALOG:%1]
RequestHeader append MyVary %{mycookie}e:%{X_CACHE_CATALOG}e env=authenticated


And when catalog change, the vary also (in the second request) and the cache is updated. You can elaborate other strategies for purging vary object with this technique.

The last point is to combined Etag and Vary Header in response. IE with a Vary header don’t treat correctly Etag header and If-None-Match is never sending. So in apache remove the tag Vary and then Etag work well for all browser

Header unset Vary

Sometimes the hard part of a python application is to integrate sso because there is an unknown : what rules is defined to get the user !

In windows, apache mod_sspi or enfold proxy give to us an http header ( name X_REMOTE_USER) to deal with the active directory. This header is like that:

Domain\user

If you have one domain it’s pretty simple. Your userid is unique

But in big company there is multiple domain controler. And user is not unique ! So how retrieve an unique user id for active directory and use it in my windows python application !

The response is get UserPrincipalName with COM and NameTranslate interface.

import win32com.client
d = win32com.client.Dispatch('NameTranslate')
d.Init(3,'')
d.Set(3,'domain\\user')
userPrincipalName = d.Get(9)

Now if you use COM with zope as me , COM is not thread safe. So init the client at zope starting and lock yours calls to the API

import win32com.client
import threading
D = win32com.client.Dispatch('NameTranslate')
D.Init(3,'')
COMLOCK = threading.Lock()

And in a function use the global D

def getUserPrincipalName(sso_header):

     try:
            COMLOCK.acquire()
            D.Set(3,sso_header)
            userPrincipalName = D.Get(9)
     finally:
            COMLOCK.release()
     return userPrincipalName

Youpi , thanks win32com !!

Hello,

In a recent zope project I’m obliged to use an mssql database. So I test first an installation with a windows server os. Everything is ok when I configure the good port (1433) and open tcpip connection in sql management and stop firewall.

But when I try to connect with pymssql in unix I have some problem.
When I launch a connection to mssql I have this error:

>>> _mssql.connect("host","user", "password")
Traceback (most recent call last):
File "<console>", line 1, in ?
MssqlDatabaseException: DB-Lib error message 20009, severity 9:
Unable to connect: Adaptive Server is unavailable or does not exist
Net-Lib error during Operation now in progress Error 36 - Operation now in progress

Yahoo , I’m very happy.

In google , no good post about this problem, so I’m very desappointed. I know that pymssql on unix work with freedts which is an implementation of the protocol which use mssql.
When you install freedts you install also some utility. It’s located in bin. Some of it  named tsql. You can use it to test your connection. As pymssql is an wrapper to freedts (in unix) and if tsql don’t work I suppose pymssql also. So I try with this command and I have exactly the same results :

mac:bin yboussard$ ./tsql -H myip -p 1433 -U myuser
locale is "fr_FR.UTF-8"
locale charset is "UTF-8"
Password:
Msg 20017, Level 9, State -1, Server OpenClient, Line -1
Unexpected EOF from the server
Msg 20002, Level 9, State -1, Server OpenClient, Line -1
Adaptive Server connection failed
There was a problem connecting to the server

So I’m very again very happy and reassure that I’m in the good way.
In read documentation about freedts and we can debug connection in setting environment variable TDSDUMP=/tmp/freetds.log. So now when I launch an connection I log in freetds.log. I see in this file the tds version used by the connection and I see it was incorrect (tds version is 5) in accordance with that documentation.
So I use an another environnement variable to fix that :

export TDSVER=7.0

And miracle , everything work with tsql. So I force version of tds in my ~/.freetds.conf in global section as this

[global]
tds version = 7.0

And after that everything is ok in python. Yahoo!!. I hope that ticket will be useful for you.

Regards Youenn.

Hi,

In test.py generated by zopeSkel there is an mistake if you want to test content type.

zcml.load_config(‘configure.zcml’,youregg) don’t initialize your egg as a zope product. allowedContentTypes() method check if factory method exists and failed for your egg content type because you don’t have an instance of it in Control_Panel.

I modify today zopeSkel in order to load egg as product if it is a zope product
So normally no more headache about that now and you can test normally the results of allowedContentTypes in your tests.

Please notice that iw.recipe.pound was renamed to plone.recipe.pound. So if you have some buildouts with iw.recipe.pound please their configuration so they use plone.recipe.pound. iw.recipe.pound is not longer maintained.

New option in 0.5 release : socket path (thanks Mathieu) , fixed some doctests (Cheettah requirement), added a run script (as runzope) in bin directory (thanks to Rocky)

Regards Youenn.

Hi !

I’m pleased to announce a new version of iw.recipe.pound.
The recipe is divided now in two part:

1. A build part (iw.recipe.pound:build)
2. A config part (iw.recipe.pound:config)

The build part accept new options as localisation of libssl.
If you have again need of compilation option (to disable some module perhaps) the extra-options keyword is for you .

The config part accept now all general configuration of pound server (see man pound). You can define TimeOut globally ( a big mistake I know ! ). The recipe give choice also to define Priority and TimeOut by backend server. The bind ip adresss of http server is required now ( before it’s always 127.0.0.1 !! so if apache or squid is located in another server the recipe serves to nothing ).

Now the backends configuration looks like that :

one  127.0.0.1:80 127.0.0.1:8080 127.0.0.1:8081,1

where :
one is the name of cluster
127.0.0.1 is the bind ip adress
80 the port of pound http server

After there is a list of backend of this pound server. Each backend have this form :

<ip backend>:<port backend>[,<priority backend>,[TimeOut backend]]

Notice that if you want define timeout you must define also priority

I hope that this recipe give you satisfaction. Don’t hesitate to submit an issue at http://trac.ingeniweb.com/ if you have problem.

Regards Youenn

In this post we describe how to install and configure an apache server in front of zope with buildout.
It’s very simple. Ok let’s go

1 – Create an zope instance with buildout

If you have a paster in your system we create an buildout skeleton with it:

$ paster create -t plone3_buildout

This create an buildout.cfg witch create an zope server (listen in 8080)

$ python2.4 bootstrap.py
$ bin/buildout

Create an instance named plone

$ bin/instance start

Ok now with your favourite browser you can see in http://localhost:8080/plone
the plone instance..
We can declare a localdns (for test purpose) that says that www.testit.com go to localhost..
So http://www.testit.com:8080/plone works also.

2 – Apache serve your plone instance

It’s possible now with buildout
We add an apache server to serve our zope in http port
Edit buildout.cfg and put this config in buildout::

[buildout]
parts =
...
apachebuild
apacheconf
...
[apachebuild]
recipe = plone.recipe.apache:build
url = http://mir2.ovh.net/ftp.apache.org/dist/httpd/httpd-2.2.8.tar.gz

[apacheconf]
recipe = plone.recipe.apache:config
bind = 80
backends =
 www.testit.com:127.0.0.1:8080
zope2_vhm_map =
 www.testit.com:/plone

You have already an httpd installed in your system . No problem
delete apachebuild conf and add in apacheconf a new directive mainconfig as this

[apacheconf]
mainconfig = /etc/apache2/apache2.conf
recipe = plone.recipe.apache:config
bind = 80
backends =
 www.testit.com:127.0.0.1:10080
zope2_vhm_map =
 www.testit.com:/plone

BE CAREFUL : the user that launch buildout must write in mainconfig so give to him the good right !! Also in order to work the current apache server must have rewrite and proxy module allowed.

So now relaunch buildout::

# bin/buildout
Getting distribution for 'plone.recipe.apache'.
Got plone.recipe.apache 0.1.0.
...
apachebuild: Downloading apache tarball.
apachebuild: Compiling Apache
...
apachebuild: Unpacking and configuring
checking for chosen layout... Apache
...
Installing apacheconf.

And after, start (or restart) apache server

# sudo bin/apachectl start

and now::
http://www.testit.com/ go to your plone instance.. Viva buildout !

Hi,

zope.testrecorder is a tool to record navigation browser for doctest. In this post I explain how install it with buildout.

In your buildout.cfg :

[buildout]

parts =
  zope2
  fakezope2eggs
  ...

eggs =
  ...
  zope.testrecorder

[fakezope2eggs]
recipe = z3c.recipe.fakezope2eggs
additional-fake-eggs = ZODB3

[instance]
...

eggs =
  ...
  zope.testrecorder

zcml =
  ...
  zope.testrecorder

Relaunch buildout and restart instance and now go to
http://<zope host>:<zope port>/++resource++recorder/index.html
You are ready to record you’re doctest.