You want to make an authenticated cache with apache/squid-varnish/plone and you don’t
know how do that : it’s possible with the vary tag.

Vary header tell to proxy cache what’s headers is variant for an object for a cache.
For example if you tell to the cache that the variant is Cookie , then for a same url with different cookie value the result of the cache is different.
The Server send to the proxy (in the response) which header is considered for vary by sending Vary: list of request header name

In Cachefu, you can configure that by rule with varyExpression.

In global configuration of cache fu you can also configure an global vary header. By default this configuration is send with rule.portal_cache_settings.getVaryHeader()

You can activate or desactivate vary with the header_set configuration ( vary field).

Vary headers must be present in the request (not response) of the browser in order to be considered to be variant for the proxy cache. So we are limited with the standard header of the protocol http.

But with cookie and apache (apache is in front of squid) we can elaborate strategy to construct a vary tag more efficient.

The second aspect of the cache work is purge content when the content change.

PURGE of Vary objects is still very poorly supported in squid, and you can only purge one variant at a time and need to get the URL cached again before being able to purge another variant. So how to deal with that also ?

First , how build our vary tag ?

The trick is to construct an custom vary tag with apache.
We can to do this with RewriteRule::

RewriteCond %{HTTP_COOKIE} mycookie="([^"]+) [NC]
RewriteRule ^(.*)$ - [E=mycookie:%1]

So in this example mycookie contains the value of cookie_key

You can add a cookie for the language , a cookie for group , a cookie for a permission and so on and then construct your custom vary tag with values of this specifics cookies with mod_headers

RequestHeader append MyVary %{mycookie}e

And then the value of mycookie is considered to be variant..

If you want have a specific vary tag for anonymous you can test the presence of
__ac cookie and send a custom MyVary in this case

RewriteCond %{HTTP_COOKIE} __ac="([^"]+) [NC]
RewriteRule ^(.*)$ - [E=authenticated:1]

RequestHeader append MyVary %{mycookie}e env=authenticated
RequestHeader append MyVary anonymous env=!authenticated

So now with that you can vary cache as you want. Now how to treat the big deal of purge.

The trick is  have an image (or a ajax request or ..) in content that is never in cache. This image is serve by a browser view (in case of zope application) that set a cookie. This cookie value is added to Vary tag. So the Vary tag change if the value of this cookie change and then the content is updated (for all request).

For example we can construct a cookie with the value of the catalog change

catalog_count = pcs.getCatalogCount()
context.REQUEST.RESPONSE.appendHeader('Pragma','no-cache')
context.REQUEST.RESPONSE.appendHeader('Cache-control', 'no-cache')
cookie = context.REQUEST.cookies.get('X_CACHE_CATALOG', 0)

if cookie != str(catalog_count) :
context.REQUEST.RESPONSE.setCookie('X_CACHE_CATALOG',
catalog_count ,
path="/")
return catalog_count

And in apache we add
RewriteCond %{HTTP_COOKIE} X_CACHE_CATALOG=([^"]+) [NC]
RewriteRule ^(.*)$ - [E=X_CACHE_CATALOG:%1]
RequestHeader append MyVary %{mycookie}e:%{X_CACHE_CATALOG}e env=authenticated


And when catalog change, the vary also (in the second request) and the cache is updated. You can elaborate other strategies for purging vary object with this technique.

The last point is to combined Etag and Vary Header in response. IE with a Vary header don’t treat correctly Etag header and If-None-Match is never sending. So in apache remove the tag Vary and then Etag work well for all browser

Header unset Vary

Sometimes the hard part of a python application is to integrate sso because there is an unknown : what rules is defined to get the user !

In windows, apache mod_sspi or enfold proxy give to us an http header ( name X_REMOTE_USER) to deal with the active directory. This header is like that:

Domain\user

If you have one domain it’s pretty simple. Your userid is unique

But in big company there is multiple domain controler. And user is not unique ! So how retrieve an unique user id for active directory and use it in my windows python application !

The response is get UserPrincipalName with COM and NameTranslate interface.

import win32com.client
d = win32com.client.Dispatch('NameTranslate')
d.Init(3,'')
d.Set(3,'domain\\user')
userPrincipalName = d.Get(9)

Now if you use COM with zope as me , COM is not thread safe. So init the client at zope starting and lock yours calls to the API

import win32com.client
import threading
D = win32com.client.Dispatch('NameTranslate')
D.Init(3,'')
COMLOCK = threading.Lock()

And in a function use the global D

def getUserPrincipalName(sso_header):

     try:
            COMLOCK.acquire()
            D.Set(3,sso_header)
            userPrincipalName = D.Get(9)
     finally:
            COMLOCK.release()
     return userPrincipalName

Youpi , thanks win32com !!

Hello,

In a recent zope project I’m obliged to use an mssql database. So I test first an installation with a windows server os. Everything is ok when I configure the good port (1433) and open tcpip connection in sql management and stop firewall.

But when I try to connect with pymssql in unix I have some problem.
When I launch a connection to mssql I have this error:

>>> _mssql.connect("host","user", "password")
Traceback (most recent call last):
File "<console>", line 1, in ?
MssqlDatabaseException: DB-Lib error message 20009, severity 9:
Unable to connect: Adaptive Server is unavailable or does not exist
Net-Lib error during Operation now in progress Error 36 - Operation now in progress

Yahoo , I’m very happy.

In google , no good post about this problem, so I’m very desappointed. I know that pymssql on unix work with freedts which is an implementation of the protocol which use mssql.
When you install freedts you install also some utility. It’s located in bin. Some of it  named tsql. You can use it to test your connection. As pymssql is an wrapper to freedts (in unix) and if tsql don’t work I suppose pymssql also. So I try with this command and I have exactly the same results :

mac:bin yboussard$ ./tsql -H myip -p 1433 -U myuser
locale is "fr_FR.UTF-8"
locale charset is "UTF-8"
Password:
Msg 20017, Level 9, State -1, Server OpenClient, Line -1
Unexpected EOF from the server
Msg 20002, Level 9, State -1, Server OpenClient, Line -1
Adaptive Server connection failed
There was a problem connecting to the server

So I’m very again very happy and reassure that I’m in the good way.
In read documentation about freedts and we can debug connection in setting environment variable TDSDUMP=/tmp/freetds.log. So now when I launch an connection I log in freetds.log. I see in this file the tds version used by the connection and I see it was incorrect (tds version is 5) in accordance with that documentation.
So I use an another environnement variable to fix that :

export TDSVER=7.0

And miracle , everything work with tsql. So I force version of tds in my ~/.freetds.conf in global section as this

[global]
tds version = 7.0

And after that everything is ok in python. Yahoo!!. I hope that ticket will be useful for you.

Regards Youenn.

Hi,

In test.py generated by zopeSkel there is an mistake if you want to test content type.

zcml.load_config(‘configure.zcml’,youregg) don’t initialize your egg as a zope product. allowedContentTypes() method check if factory method exists and failed for your egg content type because you don’t have an instance of it in Control_Panel.

I modify today zopeSkel in order to load egg as product if it is a zope product
So normally no more headache about that now and you can test normally the results of allowedContentTypes in your tests.

Please notice that iw.recipe.pound was renamed to plone.recipe.pound. So if you have some buildouts with iw.recipe.pound please their configuration so they use plone.recipe.pound. iw.recipe.pound is not longer maintained.

New option in 0.5 release : socket path (thanks Mathieu) , fixed some doctests (Cheettah requirement), added a run script (as runzope) in bin directory (thanks to Rocky)

Regards Youenn.

Hi !

I’m pleased to announce a new version of iw.recipe.pound.
The recipe is divided now in two part:

1. A build part (iw.recipe.pound:build)
2. A config part (iw.recipe.pound:config)

The build part accept new options as localisation of libssl.
If you have again need of compilation option (to disable some module perhaps) the extra-options keyword is for you .

The config part accept now all general configuration of pound server (see man pound). You can define TimeOut globally ( a big mistake I know ! ). The recipe give choice also to define Priority and TimeOut by backend server. The bind ip adresss of http server is required now ( before it’s always 127.0.0.1 !! so if apache or squid is located in another server the recipe serves to nothing ).

Now the backends configuration looks like that :

one  127.0.0.1:80 127.0.0.1:8080 127.0.0.1:8081,1

where :
one is the name of cluster
127.0.0.1 is the bind ip adress
80 the port of pound http server

After there is a list of backend of this pound server. Each backend have this form :

<ip backend>:<port backend>[,<priority backend>,[TimeOut backend]]

Notice that if you want define timeout you must define also priority

I hope that this recipe give you satisfaction. Don’t hesitate to submit an issue at http://trac.ingeniweb.com/ if you have problem.

Regards Youenn

In this post we describe how to install and configure an apache server in front of zope with buildout.
It’s very simple. Ok let’s go

1 – Create an zope instance with buildout

If you have a paster in your system we create an buildout skeleton with it:

$ paster create -t plone3_buildout

This create an buildout.cfg witch create an zope server (listen in 8080)

$ python2.4 bootstrap.py
$ bin/buildout

Create an instance named plone

$ bin/instance start

Ok now with your favourite browser you can see in http://localhost:8080/plone
the plone instance..
We can declare a localdns (for test purpose) that says that www.testit.com go to localhost..
So http://www.testit.com:8080/plone works also.

2 – Apache serve your plone instance

It’s possible now with buildout
We add an apache server to serve our zope in http port
Edit buildout.cfg and put this config in buildout::

[buildout]
parts =
...
apachebuild
apacheconf
...
[apachebuild]
recipe = plone.recipe.apache:build
url = http://mir2.ovh.net/ftp.apache.org/dist/httpd/httpd-2.2.8.tar.gz

[apacheconf]
recipe = plone.recipe.apache:config
bind = 80
backends =
 www.testit.com:127.0.0.1:8080
zope2_vhm_map =
 www.testit.com:/plone

You have already an httpd installed in your system . No problem
delete apachebuild conf and add in apacheconf a new directive mainconfig as this

[apacheconf]
mainconfig = /etc/apache2/apache2.conf
recipe = plone.recipe.apache:config
bind = 80
backends =
 www.testit.com:127.0.0.1:10080
zope2_vhm_map =
 www.testit.com:/plone

BE CAREFUL : the user that launch buildout must write in mainconfig so give to him the good right !! Also in order to work the current apache server must have rewrite and proxy module allowed.

So now relaunch buildout::

# bin/buildout
Getting distribution for 'plone.recipe.apache'.
Got plone.recipe.apache 0.1.0.
...
apachebuild: Downloading apache tarball.
apachebuild: Compiling Apache
...
apachebuild: Unpacking and configuring
checking for chosen layout... Apache
...
Installing apacheconf.

And after, start (or restart) apache server

# sudo bin/apachectl start

and now::
http://www.testit.com/ go to your plone instance.. Viva buildout !

Hi,

zope.testrecorder is a tool to record navigation browser for doctest. In this post I explain how install it with buildout.

In your buildout.cfg :

[buildout]

parts =
  zope2
  fakezope2eggs
  ...

eggs =
  ...
  zope.testrecorder

[fakezope2eggs]
recipe = z3c.recipe.fakezope2eggs
additional-fake-eggs = ZODB3

[instance]
...

eggs =
  ...
  zope.testrecorder

zcml =
  ...
  zope.testrecorder

Relaunch buildout and restart instance and now go to
http://<zope host>:<zope port>/++resource++recorder/index.html
You are ready to record you’re doctest.

Hi !

I work at the present time to an project with a lot data provided by an ldap server. The schema of this ldap is really elaborate . To reproduce it in a real ldap server is very difficult. How can I test my terrific code witch consist on an specific vocabularies and a set of PAS plugins ?

The response : a mock ldap object that give to my eggs a real API for ldap.

You can found it here : http://products.ingeniweb.com/catalog/iw.mock.ldap

The goal of this egg : testing component witch use ldap server without ldap. But also provide some good data in order to have some good test. So iw.mock.ldap read an real ldif file and you can search via ldap api (search_s).

It’s not perfect but for me I found very easy to test ldap component.

So how use it ?

In your buildout

[buildout]
eggs =
…
iw.mock.ldap

[zinstance]

eggs =
…
iw.mock.ldap

zcml =
…
iw.mock.ldap

After in you’re project where you want have some test you create a structure like that::

path-to-your-egg/mock
path-to-your-egg/mock/__init__.py
path-to-your-egg/mock/data.ldif

In __init__.py you put this boiler code::

import sys

from zope.interface import implements
from zope.component import adapts
from zope.component import provideAdapter

from iw.mock.ldap.interfaces import ILdifFile
from iw.mock.ldap.interfaces import ILdifReader
import iw.mock

class LdifFile(object):

implements(ILdifFile)
adapts(ILdifReader)

def __init__(self, context):
self.context = context

def open(self):
return file(os.path.join(os.path.dirname(__file__),’data.ldif’))

provideAdapter(LdifFile)

sys.path.insert(0, os.path.dirname(iw.mock.__file__))
from iw.mock import ldap
reload(ldap)

You can also register your adapter in an zcml (example test.zcml)

<configure xmlns=”http://namespaces.zope.org/zope”>
<adapter factory=”.mock.LdifFile”/>
</configure>

And now in you’re test you import path-to-your-egg/mock and all connections and request ldap pass
by iw.mock.ldap and use your data in data.ldif

I’m pleased to announce a new release of iw.recipe.squid. It’s aimed to configure easily an squid proxy server with zope. It’s does the same things of squid generator in cache fu product which generate some config for squid in order to work with zope. I was really inspired of this. But it work’s with buildout with some difference. So what’s else ? How configure squid with zope and apache ? Really simple (I hope)

1- install a squid and an apache server

2- you create an buildout.cfg with some config

[buildout]
parts = squid

[squid]
recipe=iw.recipe.squid
squid_accelerated_hosts =
www.mysite.com localhost:8080/myplone

www.mysite.com is the dns host name of your plone site
myplone is the plone instance name of your plone
localhost is the host name of zope server location
8080 is the port of your zope server

3- you launch buildout

> buildout -c buildout.cfg

this create several directory with some config in their.
in parts/squid/etc you have all config to work with squid
in parts/squid/apache you have all config to work with apache server
in parts/squid/log you have all log file for apache virtual host and squid acces log
in bin you have a squid controler script witch work with parts/squid/etc/squid.conf specific config

5 – finally

You must create a cache for squid

> bin/squidctl createswap

And launch your squid

> bin/squidctl start

After make symbolic link in your apache configuration location ie:

ln -s <absolute path to parts/squid/apache/*> <absolute path to apache virtual host>

And now restart apache and you have an apache/squid/zope architecture working (I hope).

The future

I’m planed to give more flexibility to the recipe.

Imagine you have 10 plone site in your zope server and you want to configure their with iw.recipe.squid . You must declare an dns name for all your plone site. Not always easy !!
So the next step is the capibility to iw.recipe.squid to deal with this use case.

The config MUST change to this way :

www.mysite.com/ localhost:8080/myplone
www.mysite.com/site2 localhost:8080/myplone2
www.mysite.com/site/myplone localhost:8080/myplone3

so when I take a browser and I type http://www.mysite.com/site2 I go to myplone2 instance without declare a specific dns name for it.