Planet Plone - Where Developers And Integrators Write

Configuring the ufw firewall to allow Cloudflare IP addresses

Posted by T. Kim Nguyen on January 03, 2018 01:56 AM

I have a Linode running Ubuntu 16.04, and I use the ufw firewall.

I have a web site running on that server, originally accessible via HTTPS on port 443 from anywhere on the internet.

The domain for that web site is managed via Cloudflare. I want the site to be available only through the domain, and not via the Linode's IP address.

Cloudflare publishes the IP addresses it uses to access your web site: https://www.cloudflare.com/ips/

Here is a page describing the overall idea of using ufw to allow access to your web site only from those Cloudflare IP addresses: https://www.ajsalkeld.com/blog/tutorial/2016/08/01/how-to-use-ufw-to-whitelist-cloudflare-ips-ubuntu-debian-digitalocean/

In this repo https://github.com/Paul-Reed/cloudflare-ufw there is a script that does this: https://github.com/Paul-Reed/cloudflare-ufw/blob/master/cloudflare-ufw.sh

I modified it a bit so that:

  • it uses the /tmp directory
  • it uses a unique filename (containing the current process ID) when retrieving the Cloudflare IP addresses
  • it specifically allows connections only on port 443 (you may want to allow connections on port 80 as well or instead)
  • it just outputs to the screen the commands that it would issue using ufw; If the commands look sane/good to you, copy and paste them into your terminal to run them

Here is my script:

#!/bin/sh
cd /tmp
wget https://www.cloudflare.com/ips-v4 -O ips-v4-$$.tmp
wget https://www.cloudflare.com/ips-v6 -O ips-v6-$$.tmp

for cfip in `cat ips-v4-$$.tmp`; do echo "ufw allow from $cfip to any port 443 proto tcp"; done
for cfip in `cat ips-v6-$$.tmp`; do echo "ufw allow from $cfip to any port 443 proto tcp"; done

Once I ran the script and copied and pasted its output into a terminal, ufw was configured as follows:

# ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22                         ALLOW IN    Anywhere
[ 2] 443/tcp                    ALLOW IN    103.21.244.0/22
[ 3] 443/tcp                    ALLOW IN    103.22.200.0/22
[ 4] 443/tcp                    ALLOW IN    103.31.4.0/22
[ 5] 443/tcp                    ALLOW IN    104.16.0.0/12
[ 6] 443/tcp                    ALLOW IN    108.162.192.0/18
[ 7] 443/tcp                    ALLOW IN    131.0.72.0/22
[ 8] 443/tcp                    ALLOW IN    141.101.64.0/18
[ 9] 443/tcp                    ALLOW IN    162.158.0.0/15
[10] 443/tcp                    ALLOW IN    172.64.0.0/13
[11] 443/tcp                    ALLOW IN    173.245.48.0/20
[12] 443/tcp                    ALLOW IN    188.114.96.0/20
[13] 443/tcp                    ALLOW IN    190.93.240.0/20
[14] 443/tcp                    ALLOW IN    197.234.240.0/22
[15] 443/tcp                    ALLOW IN    198.41.128.0/17
[16] 22 (v6)                    ALLOW IN    Anywhere (v6)
[17] 443/tcp                    ALLOW IN    2400:cb00::/32
[18] 443/tcp                    ALLOW IN    2405:8100::/32
[19] 443/tcp                    ALLOW IN    2405:b500::/32
[20] 443/tcp                    ALLOW IN    2606:4700::/32
[21] 443/tcp                    ALLOW IN    2803:f800::/32
[22] 443/tcp                    ALLOW IN    2c0f:f248::/32
[23] 443/tcp                    ALLOW IN    2a06:98c0::/29

I tested by browsing to my web site's domain (e.g. https://mysite.com) and it worked. Then I tried to browse to my server's IP address (e.g. https://123.45.67.89) and it did not work, as expected and as intended.


Update: January 3, 2018: Thank you to Florian Schulze who suggested the use of Cloudflare's authenticated origin pulls, described at https://support.cloudflare.com/hc/en-us/articles/204494148-Setting-up-NGINX-to-use-TLS-Authenticated-Origin-Pulls. With this method, you don't have to worry that Cloudflare may have changed its IP addresses (the reason why you would need to update your ufw rules periodically).

There is also TLS client side authentication, a feature described at https://support.cloudflare.com/hc/en-us/articles/115000088491-Cloudflare-TLS-Client-Auth. It is, however, available only to Enterprise Cloudflare customers.

Continuous Performance Analysis with Lighthouse and Jenkins

Posted by kitconcept GmbH on December 22, 2017 06:11 AM

Lighthouse is an open-source, automated tool for improving the quality of web pages by Google. It measures the performance of a website and provides metrics for accessibility, best practices for modern web apps, search engine optimization, and assess web applications for adherence to Progressive Web App standards.

Lighthouse Logo Lighthouse Logo

Together with WebPageTest and Google Page Speed Insights it is an indispensable tool to optimize your website performance.

Installation

Lighthouse can be installed in any JavaScript-based project by just running ‘npm install’:

$ npm install lighthouse -g

If you don’t have a package.json in your project, just install npm and run ‘npm init’ before installing.

Running Lighthouse

You can check the performance of any website by calling the ‘lighthouse’ command with the URL of the website you want to test. Append the --view parameter to show the HTML report, right after the command has finished:

$ lighthouse https://kitconcept.com --view

The report will give you five different ratings about PWA, performance, accessibility, performance best practices, and SEO.

Lighthouse Results Lighthouse Results

Continuous Performance Measurements

If you run your performance test every now and then, you always risk to hurt your website performance without noticing. If a performance regression happens unnoticed, it is usually very hard and time consuming to figure out which change caused the performance regression.

You can easily fix this and save lots of time when you run your performance tests and analysis continuously.

Unfortunately Lighthouse does not allow you to set performance test specifications that your CI system can test against, like WebPageTest or Google Page Speed Insights do (we will cover those tools in later blog posts). Though, it is still very convenient to run the performance test on a regular basis for each commit and include them into your CI report.

Install Lighthouse locally for CI

When it comes to a Continuous Integration, a local installation is prefered over a global one, which is usually harder to manage and to maintain. Especially if you have multiple projects with different sets of package versions on your CI.

Therefore we install Lighthouse locally in our project directory:

$ npm install lighthouse --save-dev

This command will install Lighthouse to your local package.json file. We recommend to use yarn or npm package-lock.json to lock down the package version you are using for a repeatable and stable project build.

For convenience, we add a “lighthouse” script to our package.json:

"scripts": {
  "lighthouse:ci": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com"
}

We call the locally installed lighthouse binary and set a static output path (by default, Lighthouse creates a file with the current date/time in the filename which makes it harder to publish on your CI).

We also include the --quiet option and run it on headless chrome, so we don’t need to install and run an X server on our CI system.

At the end, we hard-code our project URL into the command so we do not have to type it manually each time we run this command.

Now we can just run:

$ npm run lighthouse:ci

and it will create a nice HTML report that we can publish in our CI.

Configure Lighthouse for your local development environment

For convenience, we also add a command that you can run locally:

"scripts": {
  "lighthouse": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com/blog"
}

The --view parameter will fire up a browser with the report at the end of the performance analysis. This is something we clearly don’t want on our CI system.

Publish Lighthouse Reports in Jenkins CI

Travis and other lightweight CI system usually lack the option to publish any reports except the command line output. Though, if you are using Jenkins CI, you can use the HTML publisher plugin to publish your Lighthouse report.

sh 'npm install'
sh 'npm run lighthouse'
publishHTML (target: [
  allowMissing: false,
  alwaysLinkToLastBuild: false,
  keepAll: true,
  reportDir: '.',
  reportFiles: 'lighthouse-report.html',
  reportName: "Lighthouse"
])

After adding publishHTML to your Jenkins pipeline, you will see a “Lighthouse” link under the ‘Artifacts’ tab:

Link to Lighthouse report in Jenkins Link to Lighthouse report in Jenkins

There is a caveat though. Jenkins 1.641 / 1652.3 introduce the Content-Security-Policy header to static files served by Jenkins. The default header is set to a very restrictive set of permissions to protect Jenkins users from malicious HTML/JS files in workspaces.

To allow Jenkins to display the Lighthouse reports, we have to add the following JAVA_ARGS to the Jenkins startup (for instance by adding the following line to your /etc/default/jenkins file):

JAVA_ARGS="-Dhudson.model.DirectoryBrowserSupport.CSP=\"sandbox
allow-scripts; default-src 'unsafe-inline'; img-src * data:\""

For more details see the Content Security Policy Reference and the Jenkins docs on configuring Content Security Policy.

After you fixed the Content Security Policy of your Jenkins you will see the full report when clicking on the ‘Lighthouse’ link on the ‘Artifacts’ tab on your Jenkins build:

Lighthouse full report in Jenkins Lighthouse Report in Jenkins

Jenkins Declarative Pipeline Stage for Performance Tests

A full declarative pipeline stage for lighthouse looks like this:

stage('Performance Tests') {
  agent {
    label 'master'
  }
  when {
    branch 'master'
  }
  steps {
    deleteDir()
    checkout scm
    sh 'npm install'
    sh 'npm run lighthouse'
  }
  post {
    always {
      publishHTML (target: [
        allowMissing: false,
        alwaysLinkToLastBuild: false,
        keepAll: true,
        reportDir: '.',
        reportFiles: 'lighthouse-report.html',
        reportName: "Lighthouse"
      ])
    }
  }
}

We run the performance test stage on ‘master’ agents and only on the master branch. The steps performed are a simple “npm install” to set up the project build and then we run ‘npm run lighthouse’ to produce the HTML report. If you already have an npm build from a previous step you can of course just unstash the build artifact.

Jenkins pipeline with lighthouse performance tests Jenkins pipeline with Lighthouse performance tests stage

Summary

Lighthouse is a valuable and indispensable tool if you want to deliver a fast and user friendly website. Running the analysis on a continuous basis on your CI is a good idea if you take performance seriously. Setting it up is fast and easy. Maybe in the future Lighthouse will also provide a testspec feature that will allow us to fail a CI build (or mark it as unstable) on performance regressions. Though, if you run WebPageTest or Google Page Speed Insights additionally, this is not really needed.

Jazkarta Sponsors Northwest Youth Leadership Summit

Posted by Jazkarta Blog on December 07, 2017 08:21 PM

NWYLS Group Shot

Jazkarta is pleased to have recently sponsored the North Cascades Institute‘s Northwest Youth Leadership Summit. This event is intended to empower Cascadia’s future leaders in conservation by:

  • Enhancing their skills in preparation for job and college applications
  • ​Connecting with regional environmental organizations and businesses to learn about jobs and internships
  • Learning from like-minded peers about career options available in the conservation, outdoor and environmental fields

More than 220 students participated and are now better equipped to take action towards conservation. The Summit was free to all participants to ensure that underrepresented youth are given opportunities to get involved in the outdoor and environmental fields.

The sponsorship added another dimension to our existing partnership with North Cascades Institute. Just before the summit, we had given the non-profit’s Plone+Salesforce website a mobile refresh to make it work smoothly on phones and tablets. If we say so ourselves, the results are quite beautiful. Kudos to Neal Maher for the designs and to the Jazkarta team (Christine Winckler and David Glick) for a smooth implementation.

North Cascades Institute is not the only environmental non-profit organization that Jazkarta is working with – we created The Mountaineers‘s website and the Washington Trails Association ‘s volunteer management system. Both organizations were involved in the Summit. It was hosted at The Mountaineers’ Seattle Program Center, here is one of the students using the climbing wall.

NWYLS Student on The Mountaineers Climing Wall

Andrew Pringle of the Washington Trails Association led a breakout session titled “Trip Planning 101: An Introduction to Leading Backcountry Adventures”, and both organizations ran booths, talking with participants about activities, internships and employment options for young outdoor leaders.  Here’s Andrew at the WTA booth.

WTA's Andrew Pringle at the NWYLS

We feel very lucky to be helping all of these organizations further their missions.

 

— Photos by North Cascades Institute staff


Tagged: conservation, environment, north cascades institute, pacific northwest, sponsorship, the mountaineers, wta, youth

20171128

Posted by PLONE.ORG on November 28, 2017 12:00 AM
Several XSS and redirect fixes, and a sandbox escape fix.

Security patch released 20171128

Posted by PLONE.ORG on November 28, 2017 12:00 AM
This is a routine patch with our standard 14 day notice period. There is no evidence that the issues fixed here are being used against any sites.

CVE numbers not yet issued.

Versions Affected: All supported Plone versions (4.x, 5.x). Previous versions could be affected but have not been tested.

Versions Not Affected: None.

Nature of vulnerability: Low severity, no data exposure or privilege escalation for anonymous users.

The patch was released at 2017-11-28 15:00 UTC.

Installation

Full installation instructions are available on the HotFix release page.

Standard security advice

  • Make sure that the Zope/Plone service is running with minimum privileges. Ideally, the Zope and ZEO services should be able to write only to log and data directories. Plone sites installed through our installers already do this.
  • Use an intrusion detection system that monitors key system resources for unauthorized changes.
  • Monitor your Zope, reverse-proxy request and system logs for unusual activity.
  • Make sure your administrator stays up to date, by following the special low-volume Plone Security Announcements list via email, RSS and/or Twitter

These are standard precautions that should be employed on any production system, and are not tied to this fix.

Extra Help

If you do not have in-house server administrators or a service agreement for supporting your website, you can find consulting companies at plone.com/providers

There is also free support available online via the Plone forum and the Plone chat channels.

Q: When will the patch be made available?A: The Plone Security Team will release the patch at 2017-11-28 15:00 UTC.

Q. What will be involved in applying the patch?A. Patches are made available as tarball-style archives that may be unpacked into the products folder of a buildout installation and as Python packages that may be installed by editing a buildout configuration file and running buildout. Patching is generally easy and quick to accomplish.

Q: How were these vulnerabilities found?A: The vulnerabilities were found by users submitting them to the security mailing list.

Q: My site is highly visible and mission-critical. I hear the patch has already been developed. Can I get the fix before the release date? A: No. The patch will be made available to all administrators at the same time. There are no exceptions.

Q: If the patch has been developed already, why isn't it made available to the public now? A: The Security Team is still testing the patch against a wide variety of configurations and running various scenarios thoroughly. The team is also making sure everybody has appropriate time to plan to patch their Plone installation(s). Some consultancy organizations have hundreds of sites to patch and need the extra time to coordinate their efforts with their clients.

Q: How does one exploit the vulnerability?A: This information will not be made public until after the patch is made available.

Q: Is my Plone site at risk for this vulnerability? How do I know if my site has been exploited? How can I confirm that the hotfix is installed correctly and my site is protected?

A: Details about the vulnerability will be revealed at the same time as the patch.

Q: How can I report other potential security vulnerabilities?

A: Please email the Plone Security Team at security@plone.org rather than publicly discussing potential security issues.

Q: How can I apply the patch without affecting my users?

A: Even though this patch does NOT require you to run buildout, you can run buildout without affecting your users. You can restart a multi-client Plone install without affecting your users; see http://docs.plone.org/manage/deploying/processes.html  

Q: How do I get help patching my site?

A: Plone service providers are listed at plone.com/providers  There is also free support available online via the Plone forum and the Plone chat channels

Q: Who is on the Plone Security Team and how is it funded?

A: The Plone Security Team is made up of volunteers who are experienced developers familiar with the Plone code base and with security exploits. The Plone Security Team is not funded; members and/or their employers have volunteered their time in the interests of the greater Plone community.

Q: How can I help the Plone Security Team?

A: The Plone Security Team is looking for help from security-minded developers and testers. Volunteers must be known to the Security Team and have been part of the Plone community for some time. To help the Security Team financially, your donations are most welcome at http://plone.org/sponsors

General questions about this announcement, Plone patching procedures, and availability of support may be addressed to the Plone support forums If you have specific questions about this vulnerability or its handling, contact the Plone Security Team at security@plone.org

To report potentially security-related issues, email the Plone Security Team at security@plone.org We are always happy to credit individuals and companies who make responsible disclosures.

Information for Vulnerability Database Maintainers

We will apply for CVE numbers for these issues. Further information on individual vulnerabilities (including CVSS scores, CWE identifiers and summaries) will be available at the full vulnerability list.

fail2ban configuration error fix

Posted by T. Kim Nguyen on November 26, 2017 04:09 PM

If you have this in your /etc/fail2ban/jail.local configuration file:

# "bantime" is the number of seconds that a host is banned.
bantime = 31536000 # 1 year

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime = 604800 # 7 days

and you get these errors when you restart fail2ban (service fail2ban restart):

WARNING Wrong value for 'findtime' in 'ssh'. Using default one: '600'
WARNING Wrong value for 'bantime' in 'ssh'. Using default one: '600'

change it to this (put the comment on a separate line):

# "bantime" is the number of seconds that a host is banned.
# 1 year
bantime = 31536000

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
# 7 days
findtime = 604800

This is explained in the following bug report:

fail2ban: Incorrect parsing of commented text after reading a value from config file

If you want to set a permanent ban time, use a negative number.

# "bantime" is the number of seconds that a host is banned.
# permanent ban
bantime = -1

Pastanaga Sprint Bonn 2017

Posted by kitconcept GmbH on November 23, 2017 05:02 PM
Pastanaga is a new user experience framework for the web, designed by Albert Casado.

pastanaga

Pastanaga was first presented in March 2017, at the Plone Open Garden in Sorrento. In July, we started with an initial implementation during the Midsummer Sprint in Jyväskylä, Finnland.

Pastanaga was also present at the recently held Plone Conference in Barcelona, where Albert gave a presentation on it. In addition, Eric Steele, the Plone release manager, gave us the opportunity to present Pastanaga to the audience during his keynote on the first day of the conference.

With all the positive feedback and energy we took from the Plone Conference, we wanted to push things further and we just couldn’t wait until our “Beethoven Sprint”, which is planned for early 2018. Therefore we decided to organize a small and focused sprint at our office in Bonn to work on the implementation of Pastanaga.

The Pastanaga Minimal Viable Product

As an Open Source community (and software engineers) with many years of experience in designing and building complex Content Management System applications, we sometimes have the tendency to try to solve all problems at once.

Over the years we encountered and solved many complex problems and when we build something new, this can be both a source of wisdom as well as a baggage that you carry around.

This sometimes led to a situation where we were over-engineering solutions, to solve all the problems that we encountered over the years at once. Enhancements sometimes stayed around for years without really becoming production ready and usable in real-world projects.

To avoid this from happening when working on implementing Pastanaga, we decided in Jyväskylä to focus on a Minimal Viable Product.

A Minimum Viable Product (MVP) is a product with just enough features to satisfy early customers, and to provide feedback for future product development. The Pastanaga MVP needs to provide what we consider the essentials of a Content Management System:

  • A site administrator can and add, edit, and delete a page

  • A user can view the created pages and navigate the site structure

In order to be usable for public facing website projects, we added two additional technical requirements:

  • The page should be fully rendered within 500 milliseconds

  • Google should be able to crawl the contents of the website

Those requirements might sound very simple, but they are actually not.

Pastanaga aims to leverage the editing experience and reduce the complexity that we took for granted over the years. We aim to simplify the user experience for the editors by getting rid of things that we got used to. For instance, adding an image to a page should be as simple as just dragging and dropping an image to the page and Plone will take care about the heavy lifting of automatically uploading and resizing the image.

You can find a list of all the user stories that we plan to implement as part of the MVP here:

Having the goals and scope for this set the only thing that was needed was a bunch of Plone devs and three days and nights of coding.

Sprint Day One

After the sprinters arrived, we started with our sprint planning session. We decided to focus on the implementation of the Pastanaga MVP and work on the other issues (e.g. plone.restapi) only if we need them for the MVP.

After the planning meeting, Rob gave us an introduction to plone-react, a ReactJS-based implementation of the Plone UI that he and Roel worked on over the past months and that we decided to use as a basis for our MVP.

We went through all components, reducers, bells and whistles of the application and discussed best practices, developer environments and developer approachability.

After that session, Rob and Victor started with the implementation of Pastanaga. Davi created a pull request that adds an uninstall profile for plone.restapi and started to learn about React. Roel started to look into a way to turn the Plone site root into a Dexterity object, something that we would need to simplify the Plone editing experience. I worked on the basic Robot Framework acceptance test setup and updated the contents of the Pastanaga github repository, which is supposed to be just an entry point for all our initiatives around Pastanaga:

Day Two

On the second day, Victor finished the login form and made the error messages work.

Rob implemented the document edit accordion menu, fixed the button styling, made plone-react use the Pastanaga icons and started to work on the toolbar.

document edit

Davi added a search widget to the header, implemented the breadcrumbs navigation and added styles for the document heading and description.

document view

Right before the wrap-up meeting of day two, Roel showed us a Plone site with a “containerish” Dexterity-based site root. We did not really expect that much progress and went to bed (some of us a lot later) still very impressed by his accomplishment.

Day Three

On day three, Rob started to work on the new Pastanaga document edit view. He made the new edit view to show multiple content items (e.g text, image, video) and allowed to change the order of those content items via drag and drop.

Davi continued to work on the header and breadcrumbs styling. Victor looked into the mobile views of our responsive design, fixed some issues with the status messages and briefly started to look into GatsbyJS (which we plan to use to implement pastanaga.io).

Summary

After three days (and nights) of hacking, we had:

A fully functional login form with error messages and password forgotten functionality:

Login

A fully functional Pastanaga Toolbar that can be collapsed or expanded. With all the menu items present and the personal toolbar functionality available:

toolbar

A view to add and edit pages with all the existing functionality:

document edit

In three sprint days, we accomplished our main goals and were able to create the first iteration of a Minimal Viable Product that we can use to build things upon. We plan to continue to work on this, use it in our current and upcoming projects, and of course: contribute back as much as we can.

Stay tuned for more updates on this soon!

Successful Google Summer of Code 2017

Posted by PLONE.ORG on November 16, 2017 05:47 PM

Google Summer of Code ("GSoC") is an annual international program open to university students in which Google awards stipends to all students who successfully complete a free and open-source software  project.

The Plone community is proud to announce four successful projects were completed for GSoC 2017. 

All five GSoC students were offered sponsorship by the Plone Foundation to travel to Barcelona for the Plone Digital Experience 2017 conference. Oshane, Mikko, Noel, and Shriyansh Agrawal (content import and export) were able to attend and present their work to enthusiastic audiences.

Cris Ewing was our new-for-2017 coordinator of the Plone community's GSoC involvement. The Plone Foundation Board expresses its gratitude to him on behalf of the entire Plone community for having managed this very important project.

We also truly appreciate the time and effort of our GSoC students and their mentors in continuing to move Plone forward.

On to 2018, for which we have already begun soliciting project ideas

Plone Foundation Officers 2017-2018

Posted by PLONE.ORG on November 15, 2017 08:25 PM

All seven Plone Foundation Board members' nominations were accepted at the Annual General Meeting held in Barcelona on October 20, 2017: 

  • Paul Roeland
  • Alexander Loechel
  • Carol Ganz
  • Chrissy Wainwright
  • Víctor Fernández de Alba
  • Philip Bauer
  • T. Kim Nguyen

At the first Board meeting of the new term on November 2, 2017, the officers of the Foundation were voted in. The officers are elected annually:

  • President: Paul Roeland
  • Vice President: Alexander Loechel
  • Secretary: Chrissy Wainwright
  • Treasurer (non-voting): Jen Myers

Apart from these official Foundation roles, there are further roles and tasks that the Board attends to:

  • Marketing lead: T. Kim Nguyen
  • Framework team liaison: Philip Bauer
  • Security team liaison & Higher Education liaison: Alexander Loechel
  • Communications/Marketing team lead: T. Kim Nguyen
  • Front End team lead: Víctor Fernández de Alba
  • Foundation Membership committee co-chairs: Érico Andrei, T. Kim Nguyen

For more information on the Plone Foundation or its board, visit plone.org/foundation, or drop an e-mail to .

Plone is an open source web content management system excelling in usability, accessibility, and versatility. The Plone Foundation is a US 501(c)3 tax-exempt organization that protects and promotes Plone.

Thank you, Barcelona!

Posted by PLONE.ORG on November 15, 2017 06:02 PM

The Plone Digital Experience Conference 2017 in Barcelona was an exhilarating success, bringing together the Plone, Python web, and modern JavaScript front end communities in the beautiful city of Barcelona. 

IMG_0056.jpg IMG_0411.jpg IMG_0507.jpg IMG_7945.jpg

Some statistics: 

  • 10 training classes
  • 6 keynotes
  • 52 presentations 
  • 180 attendees from 21 countries
  • 2 organizing companies, 18 sponsors, 4 partners
  • 70 sprinters
  • 4 Google of Summer of Code 2017 students
  • 1 truly memorable conference dinner
  • 1 official Plone band 
  • dozens of volunteers

Some artifacts:

  • Speakers' slides can be found for almost all the presentations (video recordings still to come).
  • Photos of the conference
  • Tweets during the conference 

On behalf of the Plone community, thank you 2017 organizing team!

  • Victor Fernandez de Alba
  • Ramon Navarro Bosch
  • Agata Avalo
  • Albert Casado
  • Timo Stollenwerk
  • Philip Bauer
  • Paul Roeland
  • Kim Nguyen
  • Sally Kleinfeldt
  • Mikel Larreategi
  • Eric Bréhault

IMG_0627.JPG

Security vulnerability pre-announcement: 20171128

Posted by PLONE.ORG on November 10, 2017 04:30 PM
This is a routine patch with our standard 14 day notice period. There is no evidence that the issues fixed here are being used against any sites.

CVE numbers not yet issued.

Versions Affected: All supported Plone versions (4.x, 5.x). Previous versions could be affected but have not been tested.

Versions Not Affected: None.

Nature of vulnerability: Low severity, no data exposure or privilege escalation for anonymous users.

The patch will be released at 2017-11-28 15:00 UTC.

Preparation

This is a pre-announcement of availability of this security fix. 

The security fix egg will be named Products.PloneHotfix20171128 and its version will be 1.0. Further installation instructions will be made available when the fix is released.

Standard security advice

  • Make sure that the Zope/Plone service is running with minimum privileges. Ideally, the Zope and ZEO services should be able to write only to log and data directories. Plone sites installed through our installers already do this.
  • Use an intrusion detection system that monitors key system resources for unauthorized changes.
  • Monitor your Zope, reverse-proxy request and system logs for unusual activity.
  • Make sure your administrator stays up to date, by following the special low-volume Plone Security Announcements list via email, RSS and/or Twitter

These are standard precautions that should be employed on any production system, and are not tied to this fix.

Extra Help

Should you not have in-house server administrators or a service agreement for supporting your website, you can find consulting companies at plone.com/providers

There is also free support available online via the Plone forum and the Plone chat channels.

Q: When will the patch be made available?A: The Plone Security Team will release the patch at 2017-11-28 15:00 UTC.

Q. What will be involved in applying the patch?A. Patches are made available as tarball-style archives that may be unpacked into the products folder of a buildout installation and as Python packages that may be installed by editing a buildout configuration file and running buildout. Patching is generally easy and quick to accomplish.

Q: How were these vulnerabilities found?A: The vulnerabilities were found by users submitting them to the security mailing list.

Q: My site is highly visible and mission-critical. I hear the patch has already been developed. Can I get the fix before the release date? A: No. The patch will be made available to all administrators at the same time. There are no exceptions.

Q: If the patch has been developed already, why isn't it made available to the public now? A: The Security Team is still testing the patch against a wide variety of configurations and running various scenarios thoroughly. The team is also making sure everybody has appropriate time to plan to patch their Plone installation(s). Some consultancy organizations have hundreds of sites to patch and need the extra time to coordinate their efforts with their clients.

Q: How does one exploit the vulnerability?A: This information will not be made public until after the patch is made available.

Q: Is my Plone site at risk for this vulnerability? How do I know if my site has been exploited? How can I confirm that the hotfix is installed correctly and my site is protected?

A: Details about the vulnerability will be revealed at the same time as the patch.

Q: How can I report other potential security vulnerabilities?

A: Please email the Plone Security Team at security@plone.org rather than publicly discussing potential security issues.

Q: How can I apply the patch without affecting my users?

A: Even though this patch does NOT require you to run buildout, you can run buildout without affecting your users. You can restart a multi-client Plone install without affecting your users; see http://docs.plone.org/manage/deploying/processes.html  

Q: How do I get help patching my site?

A: Plone service providers are listed at plone.com/providers There is also free support available online via the Plone forum and the Plone chat channels

Q: Who is on the Plone Security Team and how is it funded?

A: The Plone Security Team is made up of volunteers who are experienced developers familiar with the Plone code base and with security exploits. The Plone Security Team is not funded; members and/or their employers have volunteered their time in the interests of the greater Plone community.

Q: How can I help the Plone Security Team?

A: The Plone Security Team is looking for help from security-minded developers and testers. Volunteers must be known to the Security Team and have been part of the Plone community for some time. To help the Security Team financially, your donations are most welcome at http://plone.org/sponsors

General questions about this announcement, Plone patching procedures, and availability of support may be addressed to the Plone support forums If you have specific questions about this vulnerability or its handling, contact the Plone Security Team at security@plone.org

To report potentially security-related issues, email the Plone Security Team at security@plone.org We are always happy to credit individuals and companies who make responsible disclosures.

Information for Vulnerability Database Maintainers

We will apply for CVE numbers for these issues. Further information on individual vulnerabilities (including CVSS scores, CWE identifiers and summaries) will be available at the full vulnerability list.

Obstacles on the road towards Plone 2020

Posted by Starzel.de on November 10, 2017 09:45 AM

During the sprint at the Plone Conference 2017 in Barcelona, Plone achieved a major milestone towards what is often called "Plone 2020". This is basically the effort to modernize Plone's backend and achieve Python 3 compatibility. In 2020, support for Python 2.7 will officially end, hence Plone 2020.

A necessary part of that effort was to migrate Zope to Python 3, a daunting task that was only possible by a flurry of activity that combined the efforts of many stakeholders (not only the Plone Community). Learn more about that in Hanno Schlichting's talk once the video is on the website, and on many blog posts on the Gocept Blog.

Getting Plone to run on that newest version of Zope (currently Zope 4.0b2) was another story and took a lot of work (some details are in my post here. Finally in Barcelona, in a daring move we merged all the work that had been done for that PLIP https://github.com/plone/Products.CMFPlone/issues/1351 and decided that the result will be called Plone 5.2. But by that time not all tests were green (that's why it was daring). We worked hard to get the tests to pass and to fix some issues we found when testing manually.

By the way: At the same sprint we started to prepare Plone itself for Python 3 by fixing all imports to work in both Python 2 and Python 3. But that is a tale for another blog post.

So, despite out best efforts, even one week after the conference I was not yet able to fix all the tests, and so I created at ticket to track the remaining issues.

Here this story about two erroring tests in Products.CMFFormController actually begins. Here is the spoiler: I did not really solve the issue but finally worked around it. But I still think the approach I took might be of interest to some.

The two breaking tests, test_attacker_redirect and test_regression, were passing when I ran them in isolation or when I ran all test of Products.CMFFormController with ./bin/test -s Products.CMFFormController. To add insult to injury, Products.CMFFormController is basically dead code but is still used by some of our legacy ControllerPageTemplates.

So how could I find the issue since the traceback was not really helpful?

Here is the relevant part of the log from jenkins:

#### Running tests for group Archetypes ####
Running Products.Archetypes.tests.attestcase.Archetypes:Functional tests:

[...]

Running plone.app.testing.bbb.PloneTestCase:Functional tests:
  Tear down Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.101 seconds.
  Set up plone.app.testing.layers.PloneFixture in 9.722 seconds.
  Set up plone.app.testing.bbb.PloneTestCaseFixture in 2.628 seconds.
  Set up plone.app.testing.bbb.PloneTestCase:Functional in 0.000 seconds.


Error in test test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Products.CMFFormController/Products/CMFFormController/tests/testRedirectTo.py", line 97, in test_attacker_redirect
    handle_errors=False,
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/Testing/ZopeTestCase/functional.py", line 43, in wrapped_func
    return func(*args, **kw)
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/Testing/ZopeTestCase/functional.py", line 127, in publish
    wsgi_result = publish(env, start_response)
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/ZPublisher/WSGIPublisher.py", line 254, in publish_module
    with load_app(module_info) as new_mod_info:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/Testing/ZopeTestCase/sandbox.py", line 73, in load_app
    with ZPublisher.WSGIPublisher.__old_load_app__(module_info) as ret:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/ZPublisher/WSGIPublisher.py", line 220, in load_app
    app = app_wrapper()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/App/ZApplication.py", line 78, in __call__
    return connection.root()[self._name]
  File "/home/jenkins/shiningpanda/jobs/2fa08faf/virtualenvs/d41d8cd9/lib/python2.7/UserDict.py", line 40, in __getitem__
    raise KeyError(key)
KeyError: 'Application'



Error in test test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):

[...]

    raise KeyError(key)
KeyError: 'Application'

  Ran 68 tests with 0 failures, 2 errors and 0 skipped in 1.626 seconds.
Running plone.app.folder.tests.layer.plone.app.folder testing:Integration tests:
  Set up plone.app.folder.tests.layer.IntegrationFixture in 0.027 seconds.
  Set up plone.app.folder.tests.layer.plone.app.folder testing:Integration in 0.000 seconds.
  Ran 27 tests with 0 failures, 0 errors and 0 skipped in 9.033 seconds.

[...]

Tearing down left over layers:
  Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.
Total: 733 tests, 0 failures, 2 errors and 0 skipped in 3 minutes 10.739 seconds.
#### Finished tests for group Archetypes ####

What? Why does connection.root() have no Application? This makes no sense to me, and a pdb there did not help to shed light on it at all.

First I reproduced the error by testing all packages in the test group Archetypes (where the error occurs):

./bin/test \
  -s Products.Archetypes \
  -s Products.CMFFormController \
  -s Products.MimetypesRegistry \
  -s Products.PortalTransforms \
  -s Products.statusmessages \
  -s Products.validation \
  -s plone.app.folder

Then I only used the test layers that actually got set up according to the output:

./bin/test --layer Products.Archetypes.tests.attestcase.Archetypes \
           --layer Products.PortalTransforms.testing.PortalTransformsLayer \
           --layer Testing.ZopeTestCase.layer.ZopeLite \
           --layer plone.app.testing.bbb.PloneTestCase \
           -s Products.Archetypes \
           -s Products.CMFFormController \
           -s Products.MimetypesRegistry \
           -s Products.PortalTransforms \
           -s Products.statusmessages \
           -s Products.validation \
           -s plone.app.folder

That worked, I see the error. But I will not try to read 733 tests and wait for more than 3 minutes each time I think I may have fixed something!

Thus I used the divide-and-conquer strategy to figure out which combination produced the failing tests: remove half of the packages layers and see if it still fails. If they pass, try the other half. Do the same with the layers.

Remember to keep --layer plone.app.testing.bbb.PloneTestCase and -s Products.CMFFormController in order not to skip the tests that expose the issue.

It turned out that the following combination reproduced the issue:

./bin/test \
    --layer Products.Archetypes.tests.attestcase.Archetypes \
    --layer Testing.ZopeTestCase.layer.ZopeLite \
    --layer plone.app.testing.bbb.PloneTestCase \
    -s Products.Archetypes \
    -s Products.CMFFormController

Still way too many tests to have a look, most of them in Products.Archetypes. So I removed (actually, moved the .py files to some temp folder) all python tests and kept the doctests (and their setup). The only reason was that I hate doctests and consequently it must be a doctest that created trouble. I was right.

So I kept only one doctest that produced the issue by commenting out the others in test_doctest.py of Products.Archetypes.

Now I needed to find a combination of three tests from these layers that still exposed the issue. To to that, I added the option -vv to the testrunner to see the names and python path of all tests that still ran.

./bin/test --layer Products.Archetypes.tests.attestcase.Archetypes --layer Testing.ZopeTestCase.layer.ZopeLite --layer plone.app.testing.bbb.PloneTestCase -s Products.Archetypes -s Products.CMFFormController -vv
Running tests at level 1
Running Products.Archetypes.tests.attestcase.Archetypes:Functional tests:
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.157 seconds.
  Set up plone.app.testing.layers.PloneFixture in 10.252 seconds.
  Set up plone.app.testing.bbb.PloneTestCaseFixture in 1.871 seconds.
  Set up Products.Archetypes.tests.attestcase.ATTestCaseFixture in 0.647 seconds.
  Set up Products.Archetypes.tests.attestcase.Archetypes:Functional in 0.000 seconds.
  Running:
    1/1 (100.0%) /Users/pbauer/workspace/coredev/src/Products.Archetypes/Products/Archetypes/tests/traversal_4981.txt

  Ran 1 tests with 0 failures, 0 errors, 0 skipped in 0.269 seconds.
Running Testing.ZopeTestCase.layer.ZopeLite tests:
  Tear down Products.Archetypes.tests.attestcase.Archetypes:Functional in 0.000 seconds.
  Tear down Products.Archetypes.tests.attestcase.ATTestCaseFixture in 0.010 seconds.
  Tear down plone.app.testing.bbb.PloneTestCaseFixture in 0.009 seconds.
  Tear down plone.app.testing.layers.PloneFixture in 0.065 seconds.
  Tear down plone.testing.z2.Startup in 0.004 seconds.
  Tear down plone.testing.zca.LayerCleanup in 0.001 seconds.
  Set up Testing.ZopeTestCase.layer.ZopeLite in 0.009 seconds.
  Running:
    1/5 (20.0%) test_parseXML_empty (Products.CMFFormController.tests.test_exportimport.CMFFormControllerImportConfiguratorTests)
    2/5 (40.0%) test_parseXML_with_info (Products.CMFFormController.tests.test_exportimport.CMFFormControllerImportConfiguratorTests)
    3/5 (60.0%) test_action_not_unicode (Products.CMFFormController.tests.test_exportimport.Test_importCMFFormController)
    4/5 (80.0%) test_normal (Products.CMFFormController.tests.test_exportimport.Test_importCMFFormController)
    5/5 (100.0%) test_partial (Products.CMFFormController.tests.test_exportimport.Test_importCMFFormController)

  Ran 5 tests with 0 failures, 0 errors, 0 skipped in 0.023 seconds.
Running plone.app.testing.bbb.PloneTestCase:Functional tests:
  Tear down Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.092 seconds.
  Set up plone.app.testing.layers.PloneFixture in 7.227 seconds.
  Set up plone.app.testing.bbb.PloneTestCaseFixture in 2.087 seconds.
  Set up plone.app.testing.bbb.PloneTestCase:Functional in 0.000 seconds.
  Running:
    1/4 (25.0%) testCopy (Products.CMFFormController.tests.testCopyRename.TestCopyRename)
    2/4 (50.0%) testRename (Products.CMFFormController.tests.testCopyRename.TestCopyRename)
    3/4 (75.0%) test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)


Error in test test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/pbauer/workspace/coredev/src/Products.CMFFormController/Products/CMFFormController/tests/testRedirectTo.py", line 97, in test_attacker_redirect
    handle_errors=False,
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 43, in wrapped_func
    return func(*args, **kw)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 127, in publish
    wsgi_result = publish(env, start_response)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 254, in publish_module
    with load_app(module_info) as new_mod_info:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/sandbox.py", line 73, in load_app
    with ZPublisher.WSGIPublisher.__old_load_app__(module_info) as ret:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 220, in load_app
    app = app_wrapper()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/App/ZApplication.py", line 78, in __call__
    return connection.root()[self._name]
  File "/Users/pbauer/workspace/coredev/bin/../lib/python2.7/UserDict.py", line 40, in __getitem__
    raise KeyError(key)
KeyError: 'Application'

    4/4 (100.0%) test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)


Error in test test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/pbauer/workspace/coredev/src/Products.CMFFormController/Products/CMFFormController/tests/testRedirectTo.py", line 71, in test_regression
    handle_errors=False,
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 43, in wrapped_func
    return func(*args, **kw)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 127, in publish
    wsgi_result = publish(env, start_response)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 254, in publish_module
    with load_app(module_info) as new_mod_info:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/sandbox.py", line 73, in load_app
    with ZPublisher.WSGIPublisher.__old_load_app__(module_info) as ret:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 220, in load_app
    app = app_wrapper()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/App/ZApplication.py", line 78, in __call__
    return connection.root()[self._name]
  File "/Users/pbauer/workspace/coredev/bin/../lib/python2.7/UserDict.py", line 40, in __getitem__
    raise KeyError(key)
KeyError: 'Application'


  Ran 4 tests with 0 failures, 2 errors, 0 skipped in 0.403 seconds.
Tearing down left over layers:
  Tear down plone.app.testing.bbb.PloneTestCase:Functional in 0.000 seconds.
  Tear down plone.app.testing.bbb.PloneTestCaseFixture in 0.010 seconds.
  Tear down plone.app.testing.layers.PloneFixture in 0.068 seconds.
  Tear down plone.testing.z2.Startup in 0.007 seconds.
  Tear down plone.testing.zca.LayerCleanup in 0.001 seconds.

Tests with errors:
   test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
   test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Total: 10 tests, 0 failures, 2 errors, 0 skipped in 24.082 seconds.

24 seconds? I can work with that.

Still, I removed tests from each layer until I only had three tests left and reverted my changes to Products.Archetypes.

The result is the following:

./bin/test \
    --layer Products.Archetypes.tests.attestcase.Archetypes \
    --layer Testing.ZopeTestCase.layer.ZopeLite \
    --layer plone.app.testing.bbb.PloneTestCase \
    -s Products.Archetypes \
    -s Products.CMFFormController \
    -t test_parseXML_empty \
    -t traversal_4981 \
    -t test_attacker_redirect \
    -vv

Since more than one test still exposed the issue, I kept only very simple ones because I guessed that the issue is actually in the setup or teardown.

So next I changed the test test_parseXML_empty to a simple return. The error is still there. Trying the same with traversal_4981 makes it go away.

At this point I could skip reducing the layers since I only run three tests from two packages.

It was time to actually read what the remaining tests are doing. I stripped down all tests and their setup to the base minimum that still breaks the test run and could not find anything. I turn edCMFFormControllerImportConfiguratorTests into a ZopeTestCase and a PloneTestCase and realized that the error disappears when it is a PloneTestCase. Bad. Migrating the whole test to PloneTestCase or plone.app.testing would be a lot of work since CMFFormControllerImportConfiguratorTests inherits from Products.GenericSetup.tests.common.BaseRegistryTests and does a lot of additional magic.

So the test layers for the two tests that did not fail or error by themselves but triggered the issue in the failing tests (traversal_4981 and test_parseXML_empty) seemed to be out of the scope of what I could do so I took a closer look at the failing tests themselves. I quickly found that I hate them but what they do is actually quite simple. Why do I hate them? Because they use the publish method of ZopeTestCase.Functional. That method (and its evil doctest-cousin Testing.ZopeTestCase.zopedoctest.functional.http) are way too clever helper methods that make things harder, not easier. I prefer to use restrictedTraverse or the testbrowser any time since both are much closer to what actually happens in the application.

This was the moment when I decided to migrate the tests in question to proper plone.app.testing tests. It took me about 1 hour to create a pull-request which resolves the issue. The rest of the day was spent on a fruitless attempt to find the issue that must still be lurking somewhere between the three tests and their layers.

I hope that monster will never rear its ugly head again until CMFFormController is finally removed from the coredev. The PLIP 2092 by @esteele and me will remove the last remaining ControllerPageTemplates but there are some more left in Archetypes.

I fear it will be quite some time until all ZopeTestCase and PloneTestCase tests are migrated to plone.app.testing. The remaining happy thought is that many will not need to be migrated since they are part of Archetypes and will go awaaaaay with it.

Content translation endpoint for plone.restapi

Posted by CodeSyntax on November 06, 2017 03:03 PM
plone.restapi ships with content translations support endpoint since version 1.0a22. In this post I will explain the history behind this and the decisions taken to implement it.

Plone Conference Barcelona 2017

Posted by Asko Soukka on November 03, 2017 07:35 AM

It was awesome to be back at Plone Conference this year. Finally! We have had participation in Plone conferences in 2009, 2011–2012 and 2014–2017, but for me the previous one was years ago: Plone Conference Bristol in 2014. Needless to say that I have missed the warm and welcoming atmosphere of a Plone conferences, and It's my pleasure to report that Barcelona did not let me down. Even the weather was still warm there in this October.

This year there was no single big Plone news at the conference. The latest major release of Plone CMS was released already two years ago, and the next feature release is still waiting for its gold master. Yet, there was still a lot of good news, and putting all the puzzle pieces together resulted in a clear picture of the future of Plone.

Disclaimer: These are obviously just my personal opinions on all these things Plone...

Published originally at http://tech.blog.jyu.fi/2017/10/plone-conference-barcelona-2017.html

https://4.bp.blogspot.com/-RPnuOC4AJH8/WfWkebmt4lI/AAAAAAAABLI/CRNo1l_VN2kclL7AFq7MD9poEa9-sKMFQCLcBGAs/s1600/DMsDO13WsAAh3zG.jpg%253Alarge.jpeg

Plone Conference Barcelona was so much of fun that I took a piece of it with me back home.

Plone 2020 and beyond

At first, let's say it clear that Plone CMS remains to be a safe bet for a long-term enterprise CMS solution. If there ever was any doubt, whether Plone could make it to Python 3 in-time before the end of Python 2.7 maintenance in 2020, that should be no more. Plone will make it.

All the major blockers seem to have been solved, and the rest is just hard work left for our community (check some related talks by Alexander and Hannoabout the recent events on that). Python 3 version of Zope application server powering Plone is already in beta, and it is scheduled to be released within a year. Plone, for sure, has still plenty of packages to be ported from Python 2.7 to Python 3, but there are already many sprints scheduled to continue that work in near future (including the already completed Barcelona Conference sprints). We might even have an alpha version of Plone on Python 3 before end of 2018.

In addition that, it's always good to mention, that Plone Foundation has continued to do its usual great job in all the possible paper work around managing Plone's copyrights and trademarks.

All these should be good and relaxing news for any long-term Plone user.

Let's go frontend!

The greatest challenge for Plone CMS seems to be keeping up with the ever increasing UX expections of the day, while complying with the high accessibility standards. After Plone 5 rewrote the default theme and whole front-end resource management in Plone, there are no longer blockers for using any current front-end tech with Plone. But just being able to use some tech is not enough – also the real work for better UX needs to be done. And even a lot has been done for Plone 5 and 5.1, that work seems to never end.

Plone Conference Barcelona included a great amount of front-end, user experience and accessibility related talks to educate our community. So many that I can only mention a few.

At first, there were talks regarding the current Plone user interface: Johannes gave a bit technical, but very comprehensive talk how the new frontend resource registries in Plone 5 really work. My talk instructed, how to combine the ancient powers of Zope application server with the modern Plone 5 theming support to achieve shorter iterations and faster deployments when developing new UX features. Our Rikupekka talked about our migration experiences from Plone 4 to Plone 5, and gave a demo about of the UI features we have developed using the approach I discussed in my talk. Finally, I want to mention Wildcards' Kim's talk about Castle CMS, which really showcased, how much difference well lead and focused UX development for Plone based distribution could do in just about a year. Although, the fact that Castle's development had to be forked a bit from the main Plone distribution is also telling, how difficult it is to make the same UX please everyone.

Then there were many talks about the future: there's a new branch of Plone user interfaces built completely in JavaScript on top of the great Plone REST API (which Timo gave a nice presentation about). With Plone REST API it's possible to combine the great and robust CMS features of our secure Plone backend with leading edge JavaScript based frontend. It also makes Plone based solutions feasible for the current generation of frontend developers, because only very basic Plone knowledge is needed to get started. And while there is no complete replacement of Plone user interface in JavaScript yet, there are SDK like projects with many familiar UI components already for ReactJS, Angular (check Eric's talk) and even on VueJS.

If these don't feel ambitious enough, there was one more thing: Albert'stalk about Pastanaga UI – a proposal for next generation UI for generic CMSs.

Guillotina – server for a more civilized age

I'm not sure how common mistake it is, but at least we have sometimes ended up using Plone as a framework for projects, for which Plone was not really the most optimal solution. That has happened, because Plone has some very unique features we love and trust: object database with URL traversal, extremely flexible Zope Component Architecture, and very proven security model especially designed for hierarchical data.

At Barcelona conference, Nathan from Onna presented their new ”AsyncIO REST Resource Application Server” called Guillotina (open sourced through Plone Foundation)r What makes Guillotina very special and interesting is that it has all those unique features we have learned to love in Plone ”framework”, but with minimal server footprint and first class support for asynchronous programming using Python 3 AsyncIO event loop. That should allow Guillotina to go places where no Plone has gone before.

I really hope the next year brings us a suitable project to try Guillotina in practice...

There and back again

To summarize all this, here's my picture of the future of Plone on the base of Plone Conference Barcelona 2017 in three sentences:

  • Plone CMS as we know it remains here to stay – the current users remain safe with Plone
  • Plone REST API and all the UI SDKs based on it ”save Plone” by making it a feasible solution for content management related progressive web apps
  • Guillotina ”saves Plone developers” by allowing them to transfer their current Plone ”framework” knowledge into era of high-performance Python 3 AsyncIO microservices.

Obviously there was a lot more in the conference than this. There was a lot of great talks by talented speakers. It was great to see all the old friends and make some new ones. I had a chance to meet my GSOC 2017 student Oshane Bailey. And there are no parties like parties in Plone Conferences.

Thanks once again for all the organizers. It was a pleasure to be there.

We'll see if I get to see Tokyo next year...

https://4.bp.blogspot.com/-MtyXqb6O3Yw/WfWkeV2I-tI/AAAAAAAABLM/MVy3r6Utv-MM-3z7Dxaqi28CkS0Zn_IDgCLcBGAs/s1600/DMlL1SzWsAM9mfO.jpg

Photo of me, Oshane Bailey and David Bain by Maik Derstappen. They said this pose is to honor Usain Bolt.

Plone Conference 2018 will be in Tokyo, Japan!

Posted by PLONE.ORG on October 27, 2017 03:33 PM

The annual Plone Conference will be held in Tokyo, Japan, on November 5 - 11, 2018!

Tokyo is a unique, exciting city of modern and traditional charms, and its infrastructure is rapidly evolving to welcome overseas guests for the Olympic and Paralympic Games in 2020. It will be the first Asian city to host the Plone Conference. The first PyCon APAC in Tokyo was held in 2013 and it attracted more than 500 participants. PyCon JP is an annual conference held in Tokyo since 2011 and the number of participants has been rapidly increasing.

tokyo leaves.png pycon apac tokyo.png

Organizers Manabu Terada, Takeshi Yamamoto, Zenichiro Yasuda, and Takanori Suzuki submitted the winning conference proposal, vetted by the Plone Foundation Board and announced at the Foundation's Annual General Meeting held last week in Barcelona.

Manabu Terada.png Takeshi Yamamoto.png Zenichiro Yasuda.png Takanori Suzuki.png

The conference will be promoted on Asian/Japanese media to grow the well-established Japanese Plone user base, and 1-day Conference tickets will be offered to increase local participation. Simultaneous translation will be provided for keynotes and other tracks. 

Venue

The Conference will be held at Ota City Plaza, a conference venue located in an area called Kamata. Kamata is famous for being the center of manufacturing and high technology of Japan. The venue is only 3 minutes-walk from Keikyu Kamata station and there are numerous hotels in walking distance. There are many restaurants and bars (or “Izakaya”, a Japanese pub) for local people to enjoy, so the participants can enjoy Tokyo’s nightlife at a reasonable price while indulging themselves in the local atmosphere.

Ota City Plaza.png

Two large halls will accommodate more than 300 in theater style each at the Ota City Plaza. Aside from these halls, there are 8 conference rooms which have the capacity for holding training, breakouts and tutorials. Microphone (wired and wireless), projector and screen are prepared for all halls and conference rooms.

Conference Track Themes

  • Python Web (Django, Pyramid, WSGI and more)
  • Frontend (JavaScript, Design)
  • Database (ZODB, NEO, SQLAlchemy, MySQL, PostgreSQL and more)

Training – November 5 (Mon) to 6 (Tue), 2018 (2 days)

Training will be held at the Ota City Industrial Plaza, which is also the venue for the Conference. Training will consist of 3 to 4 sessions, led by professionals who are globally known for their achievement. There will be a session in Japanese for local users. Wifi will be available for participants. 

Conference – November 7 (Wed) to 9 (Fri), 2018 (3 days)

The Conference will consist of 3 tracks. Each day's program will follow a theme. One of the keynotes will be given by a Japanese speaker known for their accomplishments (it will be simultaneously translated into English). In order to increase the number of local participants, in addition to the 3 tracks, there will be an entire track in Japanese presented by well-known Japanese speakers. 

Sprint – November 10 (Sat) to 11 (Sun), 2018 (2 days)

As always, sprints will be a part of the conference schedule and will be open for all (not limited to conference ticket purchasers). 

Overview of Tokyo

tokyo 3.png

Tokyo, Japan’s bustling capital city, is a modern, vibrant megalopolis which combines business, knowledge, creativity, and innovation. The city is the epitome of fusion where over 400 years of history and Japanese tradition juxtapose, providing a unique experience for all visitors. There is always something for everyone — visitors can choose from over 100,000 restaurants, enjoy any one of its 80 plus parks, immerse in the aesthetics of the Japanese tea ceremony, or indulge in a night of unique Japanese culture at a Kabuki theatre. For the 2016 instalment of its annual Quality of Life Survey, Monocle magazine has ranked the livability of some of the world's largest cities according to 22 metrics, with Tokyo coming out the top of the list.

tokyo 2.png

Host City of the Olympic and Paralympic Games in 2020

Tokyo will be hosting the Olympic and Paralympic Games in 2020. The city is rapidly enhancing its infrastructure, and the number of flights of both Haneda International Airport & Narita International Airport is expanding towards 2020.

Safest city in the world

Japan has a notably low violent crime rate amongst the 192 U.N. countries, according to the survey of UNODC (United Nations Office on Drugs and Crime). Visitors can walk the streets and feel completely comfortable even at night time. Many international visitors have wonderful stories of returned lost wallets with everything intact. Police boxes are scattered around the city and at most train stations and major city intersections. The Safe Cities Index 2015, compiled by the Economist, ranks Tokyo at the top in terms of digital security, health security, infrastructure and personal safety. The country is hospitable, clean, prompt, polite, and efficient and the tap water is safe for drinking. Therefore, it has never been heard that attendees of conferences in Tokyo became a victim of any crime.

Access to the venue from the airport

Tokyo provides great direct air accessibility for overseas travellers and is served by two international airports: Narita and Haneda. Narita International Airport offers over 1,610 international flights per week from 103 cities around the world, while Tokyo International Airport (Haneda) offers over 760 international flights per week from 31 major cities.

HANEDA INTERNATIONAL AIRPORT – Connecting 31 Cities

  • Largest number of flight arrivals/departures in Japan, serving 80 million passengers a year
  • Located in the city center, 25 minutes to Tokyo Station
  • Over 760 international arrivals per week
  • Connected with 31 cities around the world with plans for increase
  • 39 international airlines serve the Airport
  • 1 international terminal and 2 domestic terminals
  • 7 km from proposed venue and surrounding hotels

NARITA INTERNATIONAL AIRPORT – Connecting 103 Cities

  • 2nd largest and busiest airport in Japan, serving 39 million passengers a year
  • Connected with 103 cities worldwide
  • 88 international airlines serve the Airport
  • Three international passenger terminals
  • 80 km from the proposed venue and surrounding hotels

Access to the venue from the airport.jpg

Access to the venue from the airport chart.jpg

Accommodation

Tokyo offers more than 98,000 rooms ranging from budget accommodation to five-star luxury hotels, all assuring friendly and high-quality service to guests. There are 4,500 rooms within 15 minutes-walk or train ride from the venue. There are also Airbnb possibilities, capsule hotels and guest houses. These budget accommodations cost approximately USD 30 at the lowest.

Map of hotels around the venue.jpg

Restaurants and bars

During the Conference, lunch will be prepared, but the attendees also have plenty of choices around the venue. There are many restaurants and bars in Kamata where local people gather. Kamata is also famous for its local shopping avenue, and you can feel the vivid atmosphere of Japan’s downtown.

tokyo food 2.png tokyo food 1.png

Cost of Living

Tokyo offers quality goods and services suited for all budgets, and participants are sure to find good value for money. Although Tokyo used to be recognized as one of the most expensive cities in the world, prices are no longer high compared with other big cities. To satisfy the demanding taste buds of locals and international visitors, restaurants, cafes and bars strive to serve quality food and drinks at affordable prices.

sample lunch prices.png

sample dinner prices.png

Social Program

For the social program during the Conference, Tokyo offers various venues at reasonable prices.

food.png 

The Tokyo Metropolitan Government and TCVB will offer in-kind support, providing a Japanese Entertainment Program (i.e., Japanese Drum performance, Ninja performance, Japanese “Awaodori” Dance performance, to be decided later) at the conference social event.

ladies.png

Entry Requirements

Japan has a visa waiver agreement with 68 countries and regions, where participants can enter the country for a short-term stay with a valid passport. The following is the list of countries which are included in the visa waiver programme with Japan (this information is correct as of August 2018). The period of stay granted at the time of the landing is 90 days unless indicated otherwise. For participants who require a visa, we will provide an invitation letter to registered participants.

Visa Waiver Programme.jpg

* For details, please refer to the website of the Ministry of Foreign Affairs of Japan. http://www.mofa.go.jp/j_info/visit/visa/short/novisa.html

 

tokyo trees.png

Building instant features with advanced Plone themes

Posted by Asko Soukka on October 23, 2017 08:26 PM

Plone, ”The Ultimate Enterprise CMS”, ships with built-in batteries for building sophisticated content management solutions without writing a single line of new Python code. For example, a fresh installation of Plone allows to build custom structured content types with custom HTML views, define custom state based workflows, customize various user interface elements, and finish the user experience by configuring custom event triggered content rules to react on users' actions. Not to mention the Diazo based theming tool, which allows unlimited tweaking of the resulting HTML.

All this by just clicking and typing things through-the-web (TTW) with your browser.

Yet, still some say that Plone is a difficult to customize and extend.

The flip side of customizing Plone TTW is that it's way too easy to lost track of your customizations. That adds to technical debt and therefore cost of maintaining those customizations over years and upgrades to future Plone releases. The suggested solution to avoid those problems has long been to avoid TTW customizations altogether, in favor of customizing everything using ”buildout-installed file-system Python packages”. But that makes customizing Plone feel unnecessary difficult and technical.

At Plone Conference 2017 I gave a talk, where I showed an alternative way for this: if it was possible to bundle all those customizations together, for example in TTW managed theme, maintaining those customizations would no longer be the blocker.

Customizing Plone could be made easy again.

Requirements

Technically, Plone has supported exporting and importing most of the possible TTW customizations for more than ten years, but the user interface for that has been cumbersomely technical. Finally, Plone 4.1 introduced a new Diazo based theming feature with easy to use theming control panel and theme editor. And now, with only a couple of extra packages in your Plone setup, Plone theming features get super powers to apply site customizations with any theme.

To complete the following example, you need a Plone site with these two extra Python packages installed: collective.themesitesetup and collective.themefragments.

As usual, those can be installed by customizing and running buildout


[instance]
eggs =
...
collective.themesitesetup
collective.themefragments

or you can try out with the official Plone docker image:


$ docker run -p 8080:8080 -e PLONE_ADDONS="collective.themesitesetup collective.themefragments" plone fg

Case of the day: Wall of images

As an example feature, we build a simple folder view that displays a list of varying size images in an optimal grid layout using popular Masonry.jslayout library, with help an another library called imagesLoaded.

To summarize, building that view requires:

  • Providing JS bundles for both Masonry and imagesLoaded
  • Registering those bundles into Plone resource registry
  • A folder view template that renders images in that folder
  • Way to configure that view on a folder
  • JS code to initialize Masonry layout on that view
https://3.bp.blogspot.com/-LNyBEyLbLxE/We4-2UJN28I/AAAAAAAABKc/1W8CRGj0ykc7k1ov9zGOagl6CmxNIEqbQCLcBGAs/s1600/three-columns.png

Getting started with theming

To get a fast start, we create a dummy theme base named demotheme that simply re-uses styles and rules from Barceloneta, the default theme of Plone 5. Your theme base should contain the following files:

  • ./index.html
  • ./rules.xml
  • ./scripts.js
  • ./styles.css
  • ./manifest.cfg

At first, ./index.html is just a copy of the same theme file from Barceloneta:


<!doctype html>
<html>
<head>
<title>Plone Theme</title>
<link rel="shortcut icon" type="image/x-icon"
href="++theme++barceloneta/barceloneta-favicon.ico" />
<link rel="apple-touch-icon"
href="++theme++barceloneta/barceloneta-apple-touch-icon.png" />
<link rel="apple-touch-icon-precomposed" sizes="144x144"
href="++theme++barceloneta/barceloneta-apple-touch-icon-144x144-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="114x114"
href="++theme++barceloneta/barceloneta-apple-touch-icon-114x114-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="72x72"
href="++theme++barceloneta/barceloneta-apple-touch-icon-72x72-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="57x57"
href="++theme++barceloneta/barceloneta-apple-touch-icon-57x57-precomposed.png" />
<link rel="apple-touch-icon-precomposed"
href="++theme++barceloneta/barceloneta-apple-touch-icon-precomposed.png" />
</head>
<body>
<section id="portal-toolbar">
</section>
<div class="outer-wrapper">
<header id="content-header">
<div class="container">
<header id="portal-top">
</header>
<div id="anonymous-actions">
</div>
</div>
</header>
<div id="mainnavigation-wrapper">
<div id="mainnavigation">
</div>
</div>
<div id="hero" class="principal">
<div class="container">
<div class="gigantic">
</div>
</div>
</div>
<div id="above-content-wrapper">
<div id="above-content">
</div>
</div>
<div class="container">
<div class="row">
<aside id="global_statusmessage"></aside>
</div>
<main id="main-container" class="row row-offcanvas row-offcanvas-right">
<div id="column1-container">
</div>
<div id="content-container">
</div>
<div id="column2-container">
</div>
</main><!--/row-->
</div><!--/container-->
</div> <!--/outer-wrapper -->
<footer id="portal-footer-wrapper">
<div class="container" id="portal-footer"></div>
</footer>
</body>
</html>

Then, ./rules.xml does nothing more than includes the existing rules directly from the always available Barceloneta theme:


<?xml version="1.0" encoding="UTF-8"?>
<rules
xmlns="http://namespaces.plone.org/diazo"
xmlns:css="http://namespaces.plone.org/diazo/css"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xi="http://www.w3.org/2001/XInclude">

<!-- Import Barceloneta rules -->
<xi:include href="++theme++barceloneta/rules.xml" />

</rules>

File ./scripts.js starts empty and file ./styles.css with the following content to reuse styles from Barceloneta theme:


@import "../++theme++barceloneta/less/barceloneta-compiled.css";

.plone-breadcrumb ol {
padding: 18px 0;
font-size: 14px;
}

They both should be registered as the implicit ”theme bundle” (or ”Diazo-bundle”) in ./manifest.cfg by setting production-css and production-js attributes as follows:


[theme]
title = Demo Theme
description =
production-css = /++theme++demotheme/styles.css
production-js = /++theme++demotheme/scripts.js

Saving these files and enabling the theme should already give the basic Barceloneta experience. But let's continue to extend it with our special feature...

Registering Masonry.js bundles

Plone 5 resource registry supports many ways to configure new front end resources. We go with the easy way by simply downloading the 3rd party JS distributions and registering them mostly as such for Plone with the following steps:

  1. Create folder ./bundles into theme to keep the required front-end bundles separate from the other theme files

  2. Download the official minified Masonry.js distribution and save it as ./bundles/masonry.pkgd.min.js

  3. Download the official minified imagesLoaded distribution and save it as ./bundles/imagesloaded.pkgd.min.js

  4. Edit both of the previous files by adding line


    (function() { var require, define;

    into the beginning of the file, and line


    })();

    into the end of the file. These are required for any ”AMD packaged” JS distribution to work in Plone's Require.js based JS environment.

  5. Add two empty files ./bundles/masonry.pkgd.min.css and ./bundles/imagesloaded.pkgd.min.css for pleasing the Plone resource registry in the next step.

  6. Create folder ./install with file ./install/registry.xml with the following contents to register the above bundles into Plone resource registry:


    <?xml version="1.0"?>
    <registry>
    <records prefix="plone.bundles/imagesloaded-js"
    interface="Products.CMFPlone.interfaces.IBundleRegistry">
    <value key="depends">plone</value>
    <value key="jscompilation">++theme++demotheme/bundles/imagesloaded.pkgd.min.js</value>
    <value key="csscompilation">++theme++demotheme/bundles/imagesloaded.pkgd.min.css</value>
    <value key="last_compilation">2017-10-06 00:00:00</value>
    <value key="compile">False</value>
    <value key="enabled">True</value>
    </records>
    <records prefix="plone.bundles/masonry-js"
    interface="Products.CMFPlone.interfaces.IBundleRegistry">
    <value key="depends">imagesloaded-js</value>
    <value key="jscompilation">++theme++demotheme/bundles/masonry.pkgd.min.js</value>
    <value key="csscompilation">++theme++demotheme/bundles/masonry.pkgd.min.css</value>
    <value key="last_compilation">2017-10-06 00:00:00</value>
    <value key="compile">False</value>
    <value key="enabled">True</value>
    </records>
    </registry>

Now, once edited theme files are saved and the theme re-activated or updated, thanks to collective.themesitesetup, every response from our site should include our these new resources.

Creating a folder view with list of images

Creating a view with collective.themefragments is similar for writing any view template for Plone. Simply add a folder ./fragments into your theme with our example view ./fragments/wall_of_images.pt with the following contents:


<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
xmlns:tal="http://xml.zope.org/namespaces/tal"
xmlns:metal="http://xml.zope.org/namespaces/metal"
xmlns:i18n="http://xml.zope.org/namespaces/i18n"
lang="en"
metal:use-macro="context/main_template/macros/master"
i18n:domain="plone">
<body>
<metal:main fill-slot="main">
<metal:content-core define-macro="content-core">
<div class="wall-of-images container-fluid"
tal:define="items context/@@contentlisting">
<tal:image tal:repeat="item items">
<img tal:define="obj item/getObject;
scale_func obj/@@images;
scaled_image python:scale_func.scale('image', scale='preview')"

tal:replace="structure python:scaled_image.tag()"
tal:on-error="string:error" />
</tal:image>
</div>
</metal:content-core>
</metal:main>
</body>
</html>

Please, note, how the view template uses plone.app.contentlisting API for iterating through every item in the folder and then plone.app.imaging API for rendering image tags for scaled images. Also, note the use of tal:on-error to suppress all possible error messages (you may not always want that, though).

Enabling the view on a site

Unfortunately, collective.themefragments' views do not magically appear into Plone toolbar display menu yet. Fortunately, those views can be either be set as the default view of a content type or manually assigned to a content item by setting its layout-property:

  1. At first, let's assume that we have a folder

    http://localhost:8080/Plone/wall-of-images

  2. Then, let's open the good old properties edit form for it

    http://localhost:8080/Plone/wall-of-images/manage_propertiesForm

  3. Finally, let's add a new property of type string with name layoutand value ++themefragment++wall_of_images

Now the content should be rendered using our brand new template, displaying all the images one after one. It still does not look as intended, though, because nothing enables Masonry.js for it.

Invoking Masonry.js on the view

To enable Masonry.js on our brand new view, we could add the following code into a theme file ./scripts.js:


jQuery(function($) {
$('.wall-of-images').imagesLoaded(function() {
$('.wall-of-images').masonry({
itemSelector: 'img',
percentPosition: true
});
});
});

That code simply uses jQuery to find our view templates main element and configures Masonry.js for it after every image below it has been loaded.

An alternative for that jQuery script would be to rely on Plone's Require.js setup and define the code as a pattern:


require([
'pat-base'
], function(Base) {
'use strict';

var Masonry = Base.extend({
name: 'masonry',
trigger: '.wall-of-images',

init: function() {
var self = this;
self.$el.imagesLoaded(function() {
self.$el.masonry({
itemSelector: 'img',
percentPosition: true
});
});
}
});

return Masonry;
});

But something is still missing. Masonry.js is distributed without any default styles. To make our wall of images look as it should, we need to define responsive styles with our desired breakpoints in ./styles.css:


@media only screen {
.wall-of-images {
padding-left: 0;
padding-right: 0;
margin-top: -20px;
}
.wall-of-images img {
float: left;
width: 100%;
height: auto;
border: 5px solid transparent;
}
}

@media only screen and (min-width: 768px) {
.wall-of-images img {
float: left;
width: 50%;
height: auto;
}
}

@media screen and (min-width: 900px) {
.wall-of-images img {
float: left;
width: 33.3333333%;
height: auto;
}
}

@media screen and (min-width: 1200px) {
.wall-of-images img {
float: left;
width: 25%;
height: auto;
}
}

Finally, we'd like to make our wall of images be displayed on full browser window width. That's a bit tricky, because we need to escape Barceloneta theme's default content container, but still fully possible by adding the following Diazo rules into ./rules.xml:


<!-- Wall of Images -->
<rules css:if-content=".wall-of-images">
<!-- Make fullwidth -->
<replace css:theme=".outer-wrapper > .container"
css:content=".wall-of-images" />
<!-- Include status message -->
<before css:theme=".outer-wrapper > .container"
css:content="#global_statusmessage"
css:if-content=".wall-of-images" />
<replace css:content="#global_statusmessage">
<div id="global_statusmessage" class="container-fluid">
<xsl:apply-templates />
</div>
</replace>

Now our wall of images shines in every resolution:

https://2.bp.blogspot.com/-BvYcyG5TSaw/We4-2X7YH3I/AAAAAAAABKg/-plstMVUlqASoViy7xW9bQVPn9dC__c3wCLcBGAs/s320/four-columns.png
https://3.bp.blogspot.com/-LNyBEyLbLxE/We4-2UJN28I/AAAAAAAABKc/1W8CRGj0ykc7k1ov9zGOagl6CmxNIEqbQCLcBGAs/s320/three-columns.png
https://3.bp.blogspot.com/-IBT7ypReBsY/We4-3N3cn9I/AAAAAAAABKo/izx9FYyuItUToO0dP5AU_K18Jpm67OHlgCLcBGAs/s320/two-columns.png
https://3.bp.blogspot.com/-W7aRImzYMYE/We4-2nVw6UI/AAAAAAAABKk/f4hSBVIB5LIm8iSK6gMxijz1Ot-Ax6Y4ACLcBGAs/s320/one-column.png

PS. If want to learn more, my talk materials include a more complex example with custom content types, workflows, permissions, portlet assignments and content rules.

Summary of Plone Conference 2017

Posted by CodeSyntax on October 23, 2017 11:18 AM
It is hard to summarize an event like this year Plone Conference. The number of talks, events and trainings, and the quality of them make it hard to explain everything, but I will try to give an overview.

Sprint wrap-up Sunday

Posted by Maurits van Rees on October 22, 2017 02:04 PM

Sprint document is on Google Docs.

  • Pyramid: a few more documentation updates.
  • Plone and Zope 4. Down to seven failing tests, very good. Everything is merged, the master branch of CMFPlone is using Zope4, the PLIP job is gone.
  • Plone to Python 3. We decides to use six, which is a dependency of Zope anyway. Lots of PRs. Experimenting with sixer, which 'sixifies' the code automatically. GenericSetup: slowly working through incompatibilities.
  • Plone rest api. Some issues solved. plone.app.event stores start and end date timezone aware, and the rest of the dates are timezone naive, and there is no hint in the schema on what is naive or not, so that gives us problems, evaluating how to fix it.
  • VueJS SDK. Implementing traversal. Creating edit forms out of schema. You can add views with a plugin. Automatic testing with Travis is setup. Next: component. Editor.
  • Pastanaga Angular. plone/pastanaga-angular. Demo time! mr.developer work done.
  • Pastanaga.io, creating mocks.
  • Guillotina, made pastanaga-angular work with guillotina, you can login, browse content, navigation. guillotina_cms layer. Robot framework tests, with robotframework.guillotina for test setup.
  • Plone CLI. I can show you. Main setup is in place. plonecli create addon collective.todo; plonecli build; plonecli serve. Or in one command: plonecli create addon collective.todo build serve.
  • WSGI in plone.recipe.zope2instance. All merged. Python 3 compatible.
  • websauna. Pyramid 1.9 support is 100% done. In another week we can release a new version.
  • pas.plugins.ldap. Problem that tests are not running on Travis. We now know what is happen, but not yet why, when half a year ago it worked. We got LDAP running locally on Mac, so it becomes easier to test and fix.
  • docs.plone.org upgrade guide, just came in, documented one PLIP.
  • JSON Schema Builder with JavaScript. Demo time! You can click a form together, save it as json, and view it with Angular. From there you could save or mail the filled in data. You can do validation. We have collective.easyform which is Plone only, but this is more general: it's just json on the back end and front end. [Very impressive!]
  • Update XML-RPC to support dexterity. First pull request done.
  • Mixed bag. Removed all robot screen shots from documentation, they live under CMFPlone now, making it easier for others to write and test. Mixed results from Chrome and PhantomJS, also changing from version to version. With that, for papyrus, our documentation build system, we no longer need to build Plone.

Sprint wrap-up Saturday

Posted by Maurits van Rees on October 21, 2017 03:32 PM

Sprint document is on Google Docs.

  • Working on moving Pylons to the Plone Foundation. Tedious, painstaking work. PRs for documentation and some bugs.
  • Eric made coredev branch 5.2. Merged Zope 4 PLIP changes into that. Same amount of failures as yesterday, working on getting the build green. Work on porting databases, some mosaic problems are being fixed, most add-ons are okay. Wrote documentation for some code changes you have to do.
  • Plone to Python 3. We tried to fix all the imports in all the Plone packages that break on Python 3. Long list of PRs in the Google Doc. GenericSetup Python 3 branch that we first got to work on Python 2 again. Working through the usual string issues. Some semantic issues for PropertyManagers that we need to fix in Zope first. Gil made a list of which packages are not Python 3 yet, already in June, we ask him to update it.
  • Plone rest api. Problem with root users. There is a PR which disables that, but I have a workaround ready now.
  • VueJS SDK. plone.vuejs package, but may be renamed. Just basic stuff. Test setup. Started on some features, like traversal.
  • Pastanaga Angular. Travis setup. Universal. A mr.developer for Angular. Login form is done. Work on API and SDK.
  • Pastanaga React. Struggling with several issues.
  • Pastanaga.io, talking about license, fund raising.
  • Guillotina some work done, PR.
  • Plone CLI. Front end working. Fixing stuff in bobtemplates.
  • WSGI in plone.recipe.zope2instance. PR merged into master. Should be there in Plone 5.2. Support in the core buildout for the WSGI parts: wsgi.cfg config file. Basically done.
  • websauna. Pyramid 1.9 support is 80% done. Work on cookie cutter template to support Docker images. Will become easier to startup.
  • plone.org improvements, made mockups to make packages more visible. Set of icons will be reviewed. Should be discussed with website team. Make the listing more emotional.
  • pas.plugins.ldap. Fred chatted with Jens how we can merge back improvements from Asko and Zest. Documentation, that might be later merged to docs.plone.org. Also some collective.recipe.solr work.
  • docs.plone.org upgrade guide, worked on documenting the PLIPs, restructuring a bit
  • JSON Schema Builder with JavaScript. Browser view with drag and drop, save in dexterity object. Angular app that traverses to the end point of the schema. Missing is the order of the fields which is not correct, and actions.
  • Mixed bag. Fixes for docs.plone.org, new theme release with better version dropdown. Meeting with Manabu to talk about Tokyo. Server consolidation planning. Contributor agreements signed, 2.5 of them.

Lightning talks Friday

Posted by Maurits van Rees on October 20, 2017 04:07 PM

Andreas Jung: Collaborative content creation with smashdocs

Web based collaborative editor. Better than Google docs: it can be hosted by yourself. Intelligent documents. HTML and XML export. Tracking of changes. Chat and discussion. Docx import and export Integrates with the Plone sharing tab. Content life cycle indicator.

See https://www.creating-content-together.info

Naoki Nakanishi: Microcontrollers and Plone

I work at CMScom and I like IoT (Internet of Things). Microcontrollers can connect to Plone easily. This is because Plone has RESTful API products. We program the microcontrollers with the MicroPython language. This has the useful urequest and ujson modules. It supports many microcontrollers. I have a rough concept, but I will start to develop this from tomorrow.

Maik Derstappen: bobtemplates.plone

I have been working on bobtemplates.plone:

mrbob bobtemplates.plone:addon -O collective.todo

You can now actually add a content type in an existing package, using a sub template. It will currently overwrite code, so you want to start with a clean git checkout.

See my talk this afternoon.

Unrelated: Plone Tagung 2018 is planned on 20 March in Berlin. Main topics of this conference will be in German, but if others want to join in English, you are welcome.

Érico Andrei: several packages

  • contentrules.slack: post to a slack channel when something happens in your Plone Site.
  • collective.selectivelogin: restrict login

https://pypi.python.org/pypi/contentrules.slack/

Alexander and Sally: Plone 5 add-ons

We had nominations and votes for Plone 5 add-ons. We had problems with losing the papers where you could vote, so this is with a grain of salt. The top results:

  1. plone.restapi
  2. eea.facetednavigation
  3. plone.app.mosaic
  4. collective.easyform

On plone.org we have a list of add-ons which are managed by hand. There is a list of Plone releases, where the versions are not sorted right (alphabetically, so 1, 10, 11, 2, 3, etc). So this needs to be improved. During Google Summer of Code work was done here, getting information from PyPI. It still needs work, especially design work can help a lot, to present is nicer.

Nathan and Ramon: Docker, guillotina

Docker Compose is the new buildout? This might be a pattern that works for you.

We have a CMS on top of guillotina: https://github.com/guillotinaweb/guillotina_cms

Lots of other packages: https://github.com/guillotinaweb

Charles Beebe: Inclusion > Diversity

Inclusion is more than diversity.

Thank you all, this is my first Plone conference and I felt welcome. I never thought I would feel comfortable to do a presentation the first time I came to a conference.

Have you ever felt uncomfortable during a conference?

You may 'cover' yourself, hiding something of you. That does not help. Even 45 percent of white males in America do this. Do you make people feel at home? It does not have to be complicated. I got a cake from my colleagues when I got engaged.

Philip: Plone 2020

Plone 5.1 master branch with small changes works on Zope 4.

In Brasil Paul Everitt said: "You are dragging the dead body of Zope with you." In 2020 Python 2 is no longer supported.

We investigated and found out that Zope is actually not dead. Plone 5.2 will use Zope 4, discussed yesterday.

Plone minus Archetypes minus ZServer plus Python 3 will be some Plone version.

Some sprint will focus on this area:

  • Alpine City Sprint Innsbruck in January 2018
  • Amsterdam Spring 2018

Where we are now, felt impossible in Brasil 2013.

Roel Bruggink: demo of Plone

Plone demo, logging in, view documents, view history, view changes, edit, site setup, display menus.

What you see here, is bits of Pastanaga and bits of React front end.

Oshane: Plone theme editor

I worked on the theme editor during the GSOC (Google Summer of Code). I will give a demo. Contextual menu for renaming or moving files. Find a file by its name, or find text within files and go to that exact line. Drag and drop files. Import rapido apps.

Mikko Hursti: list customisation

I worked on improving the list customisation using mosaic during the GSOC.

See my final report.

Manabu Terada: Plone conference 2018 Tokyo

The Plone conference 2018 is going to be in Tokyo, Japan. Tokyo does not start with a B, but it has a Bay area, so is it okay?

Two years ago, we had the Plone Symposium Tokyo. PyconJP 2017 in September had lots of visitors.

FAQ:

  • English OK? Yes
  • Expensive? No, food and hotel not. Taxi, sushi, beer: a bit.
  • Safe? Yes. In 2020 we have Olympic Games.

See you next year in Tokyo, 5 to 11 November.

Ramon and Victor: Goodbye

Thank you for coming, good party, good to see new faces from other communities. I hope we keep following this path of opening up to other communities. Glad that it was safe, with all that is going on in Catalunya. We are very happy about organising this.

Thank you Agata, my beautiful wife. Thank you Timo for allowing me to spend an insane amount of time on the conference. Thank you Albert Casado for the beautiful design. Thank you Kim for all your work. Thank you to sponsors, people filling the bags, Sally, Eric, volunteers, time keepers, thanks all for joining us. It was a once in a life time experience. Hope to see you soon in the Plone world.

Éric Bréhault: Building a Cathedral Over Decades

Posted by Maurits van Rees on October 20, 2017 02:38 PM

When you build a CMS, you might start small, but you end up with a very large stack. For Plone, some of this stack is more than fifteen years old.

What do we want to work on for the future? Zope 4! Guillotina! Headless CMS! Everything! So many challenges and huge projects! In a business situation you would probably say this is bad. So why is Plone still alive? Emotions and culture.

Emotion

A software developer feels like a parent to his code. An open source community is like a shared parent group. Why does this work? Love.

Open source is not business. I can prove that. Business means you are busy. Busy means you are not free. Not free means you are not open. Clear.

The business world talks about disruption. It is violent. Okay for the business world.

Business values a 10x developer. Open source knows: the only way to be a 10x developer, is to have ten developers be twice as good.

Nine couples cannot make one baby in one month. One couple makes a baby in nine months, and it takes a village to raise the baby. Open source community.

Results versus process. Process provides emotions. Results provide money.

Developing with each other is sharing emotion. The Plone community is not just sharing code, it is sharing emotions. It feels good to share.

Empathy: feel what someone else is feeling. It is not something that you decide to do. Empathy makes it possible to share emotions. Empathy is the first open source process.

We are emotion addicts. This is true for Plone developers just as much as for Justin Bieber fans.

I think people are altruists by nature, not egoists. We want to do something for another. Our need for emotion is bigger than our need for money.

Emotion is why Plone is still alive.

Culture

Culture is how Plone is still alive.

Our everyday miracle is: pluggability. This comes at a price. Would we release a module without tests, or with a funky css selector? No. People who build Plone add-ons are following the rules, so it is safe to install.

Old Greeks had the word 'Pharmaka' for something that heals, but can also be dangerous. 'Per aspera ad astra': through difficulties to the stars. We give core commit rights to anyone who wants to join us.

The Plone community as a whole has knowledge, a diamond mine.

Building a cathedral

Plone is like the Sagrada Familia. It was created by someone who has left, and it is still being built.

Maik Derstappen: Subtemplates in bobtemplates.plone or on the way to plonecli

Posted by Maurits van Rees on October 20, 2017 01:58 PM

A long time ago, creating a Plone package was as simple as using ZopeSkel and then ZopeSkel sub templates to add for example a new content type. But ZopeSkel is dead.

Plone is using the new mr.bob already for years, with bobtemplates.plone as main template for a new Plone project. But there were only basic templates, no list of templates, no way to add a content type or other things with a sub template, a hard to remember command. So let's fix that!

My vision: give me a tool which helps for:

  • creating different projects, like an add-on or a buildout
  • extending packages with content types, vocabularies, a theme
  • provide best practice skeletons

Something like this:

$ pip instll plonecli
$ plonecli -l
templates:
 - addon
   - content_type
   - portlet
 - buildout
$ plonecli add content_type

Standalone templates:

  • addon: basic Plone addon
  • buildout: development/project buildout
  • theme_package: full stand-alone theme package, based on Barceloneta, with grunt setup

Sub templates:

  • content_type: Dexterity CT with XML or zope.schema
  • vocabulary: dynamic vocabulary
  • theme: advanced theme, including themesitesetup and themefragments
  • more to come: tile, behavior, portlet

I have a fork of mr.bob that can list templates.

On plonecli you could use shorter, more easier to remember commands, and I want autocompletion to make it even easier.

We will sprint on this, so please join.

Hanno Schlichting: Zope on Python 3

Posted by Maurits van Rees on October 20, 2017 01:30 PM

I am currently still the release manager for Zope. There is now a actually a release team.

Plone is built up like this:

  • Python is the programming language.
  • On top of this is the ZODB and the ZCA (Zope Component Architecture)
  • Then the ZTK (Zope ToolKit), just a bunch of packages.
  • Above it is Zope (2.x/4.x)
  • Then CMF
  • Then the Plone CMS

Unrelated to Plone:

  • Pyramid sits on top of the ZCA.
  • Grok sits on top of the ZTK.

The ZTK (like zope.interface and zope.component) was already mostly Python 3 compatible at the moment when we started working on compatibility for Zope. Jason Madden has done a lot and Marius Gedminas, Tres Seaver, a bunch of others.. There is very little development on the ZTK level, mostly just letting it work on newer Python versions.

Alexander Loechel already gave a talk on this conference about porting RestrictedPython.

We no longer call Zope Zope2, but just Zope. This is version 4. Forget about version 3.

ZODB and the ZTK support Python 2.7, 3.4, 3.5, 3.6, PyPy, PyPy3. Status page of Python 3 compatibility for Zope related packages: https://zope3.pov.lt/py3/ More on the ZTK: https://zopetoolkit.readthedocs.io/

Zope 4.0 beta 2 has been released. That means no new major features should be introduced, focusing on bug fixes. Zope supports Python 2.7, 3.4, 3.5, 3.6. There is no support for PyPy or PyPy3. RestrictedPython is a big reason there, because that is currently just not possible on PyPy. For Acquisition you also have to use a C implementation, so no PyPy.

On https://blog.gocept.com you can find some good reports and stories about the Zope porting sprints.

CMF 2.4 beta is released. It targets Zope 4 compatibility. Currently only Python 2.7, but progress is being made on Python 3 support. The Plone community is also busy with that.

Some other changes in Zope 4, not Python 3 related:

  • The distribution name was changed from Zope2 to Zope. Zope2 is now a meta distribution, depending only on Zope. This is similar to ZODB3 being renamed to ZODB. Please keep Zope2 as dependency in your own package, unless it really only works on Zope 4.
  • Zope now uses WSGI, and there is a separate ZServer project. ZServer was written in the days of Python 1.5, so really old. We wanted to get rid of that and use WSGI instead. But to not completely lose the old code, we made this new ZServer project. The WSGI server only supports HTTP, not for example FTP. Also not WebDav as that was really to hard, although it is built on top of HTTP. Probably ZServer is never going to be ported to Python 3. So with WSGI you use something like waitress or gunicorn or Apache mod_wsgi to call Zope.

There are more optional projects for Zope now:

  • ExternalMethod
  • PythonScripts
  • MailHost
  • TemporaryFolder
  • Sessions
  • SiteErrorLog (instead just use the standard Python logging and the WSGI solutions for logging)

Some of these have been ported to Python 3, some not.

More changes:

  • There now is a WSGI based zope.testbrowser. The old test browser was based on mechanize, which was not maintained and not ported, and that was not going to happen.
  • There is full IPv6 support in waitress.
  • Chameleon page templates is in Zope, without needing five.pt anymore.
  • zope.globalrequest is in Zope, without needing five.globalrequest anymore.
  • Upcoming minor change: support for unicode object ids under Python 3.

Zope 4 was started five or six years ago. The idea was first to remove lots of things to make it simpler. This has been partially reverted. But you might miss some features.

Future:

  • Expect more Zope 4 beta releases over the next monts.
  • Expect a final Zope 4 final release when it is done. :-) That will be when we have the feeling it is stable enough and we are only doing small bug fixes. See the issue tracker.
  • There are no plans for Zope 4.1 or 5.0 yet. A nicer ZMI (Zope Management Interface) is being worked on, so it does not look like it was made in the eighties.

Remarks from the audience:

  • David: Plone on Zope 4 is being worked on, and is making progress. We will further sprint on it this weekend.
  • Eric: We will create a branch for Plone 5.2 this weekend, and try to get an alpha out soon.

Johannes Raggam: Resource Registry Demystified

Posted by Maurits van Rees on October 20, 2017 11:34 AM

The rewrite of the RRs (resource registries) started in 2013 or 2014 and landed in Plone 5.0. I would recommend using Plone 5.1, as various things have been improved there.

With the RR you register and deploy JS and CSS. You can organise dependencies, and optimise resources and number of requests. The resources are grouped into bundles, they are concatenated and minified. Add-ons can easily register their resources. Cache headers are set automatically.

In Plone 4 there were no formally defined dependencies, so that made it hard to manage. You just had a list, and that order was used.

In Plone 5, the RRs are based on plone.registry, RequireJS, LESS and the command line interface of gulp. Instead of LESS, a lot of projects have switched to using SASS now. RequireJS is also less popular. So there is still room for advancement.

The Plone 5 way solves dependency management, but it is complexer and harder to debug. I have had my problems with it, but usually it works quite well, and is a huge improvement over Plone 4.

The js/config.js file in mockup contains configuration on how the javascript in Plone should be built. In Products/CMFPlone/static/plone.js you can see how the plone bundle is defined, and what it requires.

You can still define legacy resources, which work like they did in Plone 4. They are wrapped by some code that temporarily undefines the define and require definitions, otherwise you get errors.

In Products/CMFPlone/static/plone.less all the needed LESS definitions are imported or defined, used for creating our CSS. LESS is very handy for defining for example a text color once and use it in lots of places. [In Plone 4 we did this by using DTML files.]

You can customise and override the plone and plone-loggedin bundles in your code, if you maybe do not need everything that Plone offers by default.

collective.lazysizes has a good example of defining resources and bundles. In its registry.xml it uses condition="have plone-5" to only apply this part on install when the site is Plone 5.

With ./bin/plone-compile-resources -b plone you compile the plone bundle resources. This is also possible TTW, but I recommend the command line tool.

Future

  • Use webpack for compiling bundles. Asko Soukka has started with this for Plone. You can already use it, but it is too early for Plone core. There are some things to fix. I would like to not use RequireJS anymore, which would make it easier to use webpack; instead, ReactJS would be better. The legacy resources currently need special configuration, and that is not very expandable.
  • PLIP 1955 for RR improvements, but on hold now for lack of time and vision
  • PLIP 1653, restructure CMFPlone static resources

Devon Bernard: Lean React - Patterns for High-Performance

Posted by Maurits van Rees on October 20, 2017 10:37 AM

Get a normalised state for your json data. Think about how to structure your data so you don't duplicate data, and you have quick retrieval. For example use the normalizr library.

Use Redux development tools to give you hints or boiler plate for a new test.

Use components. When you use them, make sure they don't block other components: if four other components are not getting shown until a fifth one is ready, that is not a good idea. Give the user the information that is already there. Already show a skeleton on the page of how the component is going to look, so you only need to fill in some extra stuff and the user can already see how the component will look like.

Watch the component life cycle: which part is taking the most time?

Check if repainting is really needed before you do it: maybe a data value gets set but it is the same as the old value. Catch this and save on rendering. Use the Chrome Render Tools.

Use local, non-committed environment files to make differences between local development and production. .env may contain the default values for everyone, committed, and .env.local has your local tweaks, and you let git ignore that.

Use route wrappers to for example ease checking for anonymous or authenticated users, and do some calculations in there, so you don't need to do that in all kinds of places.

Offline first: have some javascript that runs in the background for hijacking netword requests. If you are offline, this should queue the network requests for later. IndexedDB could be more useful here than localstorage.

Use the ESLint command line utility to check the quality of your code, including your fellow developers.

Find me on Twitter: @devonwbernard.

Alexander Loechel: Modern Python Testing

Posted by Maurits van Rees on October 20, 2017 09:52 AM

I have worked in the German air force. They have rules on how to write software. For example, you start with user requirements. In open source, who is writing down user requirements? The requirements are there, but they are often implicit. Write tests that test the requirements.

'Testing leads to failure. Failure leads to understanding.'

A test is a specific set of assertions.

You can test requirements, design, interfaces, code/implementation, documentation (embedded code), conventions.

The basis for tests in Python is the unittest module. It has lots of specific assertions, like assertEqual, assertIsNotNone, etc. But why can't we simply use assert? That is what the pytest module does instead. It makes writing the test simpler. It is by now the de facto standard in the Python world. It has a pluggable add-on system.

There is unittest2, nose, nose2, but today they are mostly outdated. Use pytest.

Robot framework is used by Plone to do web testing with a real browser.

Richard Feynman: 'The first principle is that you must not fool yourself, and you are the easiest person to fool.'

For test runners we have the unittest testrunner, zope.testrunner, pytest-testrunner, gocept.pytestlayer. Such a runner collects tests for execution and shows the outcome to the user. If you use unittest, you can use these. A test runner can interact with other tools, like coverage.

On a command line you can usually run python setup.py test to run tests. Or you have scripts to run it, like buildout can install with a recipe.

You want to run your tests automatically on a test server:

  • Travis CI has Linux and MacOSX machines, and are perfect for pure Python tests
  • Circle CI has Docker containers for Linux and MacOSX
  • Appveyor tests on Windows.

On Travis you can use travis_retry in your .travis.yml file to retry a command three times. This can help when there is a temporary network problem.

tox is a test invocation tool. You use it to run your tests on multiple environments, for example Python 2 and 3, or with an extra package installed. You can use additional helpers like pyenv for virtual environments. One of the environments can apply isort, or run zest.releaser, even if these environments are not run by default. Or you can run a linter and have it report on your code quality.

My wishes for better practices in Plone:

  • Adapt tox on all packages.
  • Switch to different package structure and enforce that, like bobtemplates.plone, with docs, src, Tests.

Maybe do not ship with the tests in the PyPI releases: this code will not be run in production. Tests belong in the source distribution, but not in binary packages like wheels or eggs. The unit tests should be tested within the actual package. Integration tests could live somewhere else.

See http://plone-best-practices-discussion.readthedocs.io/

See the slides.

Nathan Van Gheem: Introduction to Python Asyncio

Posted by Maurits van Rees on October 20, 2017 09:48 AM

This is about the Python 3 core asyncio library. "This module provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives."

The first time I read that I was like: what?

Asynchronous programming using async and await syntax. Any network activity should not block other code, that is the main idea. This is useful because web applications use TCP sockets. It is a way to improve performance and scale web applications. Also think of microservices.

The optimised event loop allows you to handle a larger number of requests per second. You can have long running requests with very little performance impact. With standard Plone that is impossible.

Requirements: Python 3.4.

How are typical web servers designed like Flask, and Django? Each request is tied to a thread, so you are limited to handling number of threads and processes you run. Threads are expensive (GIL, context switching, CPU). If no threads are available, further requests are blocked, waiting for an open thread. Threads are blocked by network traffic, for example to a database server.

With asyncio, requests can be tied to tasks. You can have lots of tasks per thread, and if a task needs to wait for network traffic, it does not hurt you. But be careful: if anywhere in your code you use the requests library instead of asyncio, that will block your network traffic.

We have Futures`.  ``asyncio.run_until_complete with ensure_future wraps your asynchronous call in a Future object.

You can have long running Tasks. Tasks, futures and coroutines are very similar, in the beginning you don't need to worry about that.

Gotcha: everything must be async. Async functions need to be run by the event loop. If you run it manually, it will not do anything. If you don't call an async function using await it will never be run either.

asyncio is single threaded: only one event loop can run in a thread at a time. Running multi threaded code in asyncio is unsafe. You can have multiple threads, each having their own event loop. You can get the feel of multiprocessing by using asyncio.gather

With an 'executor' you can make synchronous code asynchronous. Typically it is a thread executor. Try to avoid it, but it is a tool that you can use if needed. See concurrent.futures.

asyncio comes with an amazing subprocess module, so you can await the result of executing a command on the terminal.

The event loop is pluggable, for example tokio.

More and more libraries are popping up using asyncio:

  • aiohttp: client and server library
  • aioes for elastic search
  • asyncpg for postgres
  • aioredis
  • aiobotocore
  • aiosmtpd for smtp

[See https://github.com/aio-libs for more.]

Debugging is more difficult than regulare sequential programs, the pdb is tricky. aioconsole allows you to have a Python prompt with an asyncio loop already setup for you.

guillotina uses asyncio.

In Python 3.7 you have an execution context, which is going to be nice.

Questions and answers:

  • You cannot do WSGI with asyncio. But Tornado uses asyncio.
  • What was hardest? Wrapping your head around it all.
  • Is this only for network calls? Or also useful for disk access? There is an add-on for that. I tried it and then it was kind of a hack.
  • Do you have profiler tools, like seeing if code is blocking too long? See an earlier talk. There is `aiomonitor <https://github.com/aio-libs/aiomonitor>`_.

Twitter: @vangheezy

Bert JW Regeer: I broke what? Taking over maintenance on existing (well loved) projects.

Posted by Maurits van Rees on October 20, 2017 08:42 AM

Existing code needs love too! Look at the truth behind open source apps on commitstrip. You can help open source projects by becoming a maintainer. I became maintainer of WebOb.

What I was told: don't mess it up. There was no existing maintainer at the time. It was handed over to the Pylons project for maintenance, but no single person was responsible for it. They all stepped back.

Side-track: imposter syndrome. See Denys Mishunov's talk yesterday. Usually you get extra responsibility gradually: for example first commit rights, then release rights. You may think you are not good enough for extra responsibility, but probably you are.

You have all these nice ideas and good intentions. You push out some alphas and betas, all seems okay. You make a full release. Then a bug report comes in because you removed something that you did not expect anyone to use.

Code grows over time. All kinds of code is there because it fixed an actual problem for an actual user. So you are faced with backwards compatibility. How much of it do you do? It depends on whether you are using a library or a tool. Libraries need to maintain more backwards compatibility. For a command line tool, the only API is the tool, not its internals.

Can you afford to lose your users? Someone can fork your code and maybe rename it and create an alternative tool.

Testing: if you have 100 percent test coverage, and you change something and a test breaks, then you can more easily see what is wrong. Does the test simply need to be rewritten for the new situation? Or is the test breakage a hint that something will break for a user, letting an old bug resurface?

You sometimes have to make a breaking change. Give a deprecation warning then.

Joel Spolsky: 'Single worst strategic mistake: rewrite the code from scratch.' This was about NetScape 6, and allowed Internet Explorer to catch up and take over. It is an option, but probably not the best.

So. You took over. You are now the gate keeper. You are a temporary guardian. Eventually someone else is going to take over. You should start looking at mentoring opportunities. Find ways to engage others, engage the community. Create pull requests instead of pushing to master.

Reach out to other communities, consumers of your code. Can you help them?

People may ask or tell you to just accept this pull request. Just push out a new version. Just is a bad word. Push back and insist on them following the standards of your project. If you require 100 percent test coverage, don't review the pull request.

I have received bad bug reports, so now I myself write better bug reports. I push myself to do better. Maybe even try to provide a fix for a bug upstream.

Be friendly when someone does crazy or seemingly stupid things in a pull request. A good question to get clarity is: 'What are you trying to accomplish?'

Twitter: @bertjwregeer.

Mark Pieszak: Rendering JavaScript on the Server? Welcome to Angular Universal.

Posted by Maurits van Rees on October 20, 2017 07:58 AM

[Sorry Mark, I came in late.]

SSR = server side rendering

Create an app.module.ts and an app.server.module.ts

  • Static SSR is done at build time.
  • Dynamic SSR is done at run time.

SSR gotchas:

  • If you use window or document, the server does not know what to do: this only lives in the browser. If you must, create a WindowService and use dependency injection to provide different versions. Use isPlatformBrowser() as much as possible. Hide things from node. Not all parts need a server version.
  • Be careful with timeouts, because they will let your server wait.

Conclusion:

  • Universal makes SEO possible.
  • Universal gives really fast initial painting of your app, and you keep the interactivity. Can be two to three times faster on mobile.
  • Be mindful of browser-specific things you might be using in your code.
  • Choose third party libraries carefully, as they need to be mindful of the pitfalls as well.
  • It takes a bit of work, but it is worth it

Further reading:

Annual membership meeting Plone Foundation

Posted by Maurits van Rees on October 19, 2017 04:11 PM

Plone Foundation president Paul Roeland presents the report of the past board year. Documents can be found on plone.org.

There were lots of sprints, most of them sponsored by the Foundation.

Eric Steele was interviewed by podcast init, which you should listen to.

At http://smile.amazon.com you can by stuff from Amazon and have Amazon give a percentage to the Plone Foundation, at no cost to you.

Sad is that long time Python organiser Jean Ferri from Brasil passed away.

Financials. Summary: we are doing fine, and can afford to spend a bit more. We would like more sponsorships, like providers on plone.org.

The entire current board nominate themselves for the new board, and there are unfortunately no other candidates, so we can have an easy vote.

Erico motions to approve the candidates. Maurits seconds this motion. Philip and Alexander abstain. Otherwise everyone says aye. The old board is hereby the new board.

Erico: Is the Foundation supporting bitcoin donations? The Free Software Foundation does.

Paul: Not at the moment. Depends on our bank account.

Matthew: Does the Foundation have an environmental policy?

Paul: We have recommendations. For this conference it does not really work, also because we have a caterer.

Paul: We may want to open up the Plone Foundation to family and friends, like guillotina and Pyramid. Pyramid lacks a legal framework currently. This needs careful reflection before we do anything, but the board is initially open and positive to it.

Paul: We are sometimes in difficult discussion with the Zope Foundation, which does not technically exist anymore, linked to the Zope Corporation, which technically does not exist. So the situation is unclear. We are working on it.

Philip: At Zope sprint in Halle there was consensus to unwind the Zope Foundation and incorporate everything in the Plone Foundation.

Alexander motions to adjourn the meeting and go party. Fred seconds. All say aye.

Thank you and have a great party. Party responsibly and be there tomorrow at nine for the keynote speaker.

Oh, there were two proposals for the next Plone conference. The board did due diligence and found that only one was viable. It will be announced tomorrow.

Lightning talks Thursday

Posted by Maurits van Rees on October 19, 2017 03:43 PM

Andreas Jung: Plone and the blockchain

Blockchain is the base technology behind bitcoin, but it is not bound to crypto currencies. It is a distributed data structure, usually based on peer to peer. No central entity of control.

Each block has a hash of its previous block, timestamp, transaction root, and a nonce.

Use cases: auditing, financial transactions, logistics, QA, legal, automotive, others.

What does this have to do with Plone and CMS? Some ideas:

  • revision safety
  • audit trail
  • verification of content integrity and authenticity

Our use case: collaborative editing environment. So we created SmashDocs. Using Plone and BigChainDb.

Erico Andrei: Websauna

Websauna is a web framework based on Pyramid. https://tokenmarket.net is created with that, also with blockchain BTW. Miko says hi!

I am using it too, and helping him. We want you on board as well, and we can sprint on it. Also we want to improve Pyramid. Move Websauna to Pyramid 1.9. Documentation. User testing.

See https://websauna.org.

Alexander, Anton: Ploneconf and PyconWeb

Why don't we make a PyconWeb conference? We did that last year. Next one starting on 9 June 1918. See you at https://pyconweb.com

Fred van Dijk: Music to your ears

Confession: I am a bit of an audiophile. When I talk with people about how they listen to music, I get sad. Lot of people use a ten euro headphone. "I don't hear a difference with more expensive ones." I am convinced that a better audio setup helps you work better.

What is the weakest link? Music source (up the settings), D/A converter (underestimated component), cables (spend twenty euros, that's the sweet spot), headphones (ten euro and you expect quality?).

If you divide the costs over the number of hours you listen music: I came at six cents an hour.

Especially the D/A converter (USB) really helped me.

Nejc Zupan: A few Pyramid goodies by Niteo

  • pyramid_force_https
  • pyramid_redirect
  • pyramid_heroku

Releases are on PyPI, enjoy!

Manuel Reinhardt: Giesing 2060

Science fiction writing project about an area of Munich in the year 2060. Using Plone, two content types, snippets that give links to other story snippets, different story lines, you can read through it in various ways.

If you like reading or writing science fiction or Python code, or both, have a look at http://giesing2060.de

Alexander Pilz: Ten years of Euphorie

A Plone success story.

This is a software to guide employers and employees for mandatory health and safety risk assessment. In 2007 Euphorie was created by Wichert Akkerman and Cornelis Kolbach, with the NuPlone interface (currently still working on Plone 5 actually). In 2008 adopted by Europe. In 2016 interest by an industry client of ours.

Why was this successful with Plone? Customisation and enhancement was made easy. Good security. Open source. Now they no longer need to cary kilos of paperwork to factories.

Jens Klein: Alpine City Sprint

I invite you to come to Innsbruck to work with us on the next Plone. Today on an open space we discussed that we may be using Zope 4 on Plone 5.2, and we can work on that.

Welcome January 2018 in Austria. We always visit a special place as well, now a space lab, with simulation for Mars missions.

Denys Mishunov: debugger; for Developers

Posted by Maurits van Rees on October 19, 2017 02:59 PM

It's been a long time since I have been in the Plone community. Good to see so many old friends! Literally old. :-)

I am Denys and I have a problem. I am a developer. Should I use Angular or React? Plone? I am not going to talk about that. I am going to talk about us as humans.

Goldman's dilemma, phrased in 1982: 'If I had a magic pill and it would let me win every match for the next five years, and then kill me, would I take it?' A lot of people would do that, for five years of success. There is no such pill, no single road to success.

As developer your life begins, you read a first book, you do a first project, you get your first job as developer, things look good. But: our program starts raising errors. When that happens, you stop, debug, and fix.

Perfectionism

One of the bugs is: perfectionism. 'Denys, your work style is like champagne. The company that we merge with, their style is more like prosecco. Less good, but at a party no one notices the difference.'

Perfectionism can be really good and bad. It can be healthy and unhealthy, positive and negative. Steve Jobs was a perfectionist. That worked out good for consumers, but he could be hard to work with, having problems picking the perfect beige color.

Several perfectionist problems:

  • Perfectionist paralysis. This can be one reason for procrastination, waiting for an ideal moment to start with an ideal project. Fear of failure: not getting a perfect result. They want perfect tasks, where they know they will succeed.
  • Picking a detail.
  • Unnecessary task. 'This can be improved. It is not hard, it would not take more than fifteen minutes.' And you spend a day on something that does not give value, or even makes things worse. Perfectionists never know when to stop.

So stop being a perfectionist? No, make your perfectionism positive. Henry Ford was a perfectionist, constantly improving the design, and never going in production. He failed at two companies before investors stopped him.

Think: 'My product should be perfect. And this release/feature/commit moves me one step closer to this perfect result.' Stephen Hawking: 'Perfection simply doesn't exist.'

Imposter phenomenon

  • You think that your success is due to luck/timing/etc.
  • You think that others might discover that you are not as skilled as they think you are
  • You think that others are more intelligent than you are.

Will Smith: 'What people think is my self confidence, is actually my fear.' Lots of people have this. Among them very successful people. Both men and women have this, also in science, shown by studies.

I read websites, Twitter, news feeds, RSS, I get a lot of information, but I cannot read everything. And I still have to work. So I stopped reading them. I may skim the titles, and open a few browser tabs, and leave them open for a few days. I did not want to read it, but my imposter syndrome wanted to read it.

What do we do about it?

  • Embrace imposterism. When you are about to read yet another news story: stop and enjoy. You are learning.
  • Measure yourself with your own yard stick. That is a good comparison.
  • Communicate your fears. Writer Neill Gaiman met Neill Armstrong. 'If he feels like an imposter, maybe everyone else does.'

Long hours

Have you ever worked long hours? Once? Regularly? Did the code work the day after?

A regret of many people before they die is: why have I worked so hard?

Long hours happen. It can be temporary, which can be normal for a hard worker. But it can also be permanent, which is a signal for a workaholic. It is hard to spot the difference. Hard working people have some balance. Are you working and thinking of skiing? Or the other way around?

If you normally work 40 hours, and you work 20 hours more, then this raises productivity for three to four weeks. After that, it drops. Why? Our brain gets tired. So we make errors. So we lack accomplishment. So we make long hours to fix those bugs. And we are in a loop.

The Japanese have a word 'karoshi': death by overwork. Same for 'Guolaosi' in Chinese: 1600 deaths a day.

When the brain detects stress, it sends signals: the chemicals adrenalin and cortisol. This couple is essential for our well being. But when applied for a prolonged period of time, things go wrong: premature ageing of our brain, drop in learning abilities, weakened memory. So it leads to mental disability: you work long hours, and you end up with dementia or alzheimer.

How do we prevent stress?

Look at Harvard University, established 1636, home to 133 nobel laureates and 8 presidents. Students asked if they could do a four year study in three years. Answer: 'Slow down. Get more out of Harvard by doing less. Take a year off.' It helped those who returned after a year. 'College can help you learn how to think, more than what to think.' We need room to think.

Our lives have only version 1.0. We need to fix bugs live, in production.

We need more healthy and happy developers. Acknowledging a problem is the first step to mental health. Debug yourself often and stay healthy and happy. Thank you.

See also my article in Smashing magazine which was published yesterday.

Trust me: at thirty five, either you start thinking about health, or health starts thinking about you.

Plone Conf 2017 Day 2: Timo Stollenwerk: Building Bridges - The Headless Future of Plone

Posted by David "Pigeonflight" Bain on October 19, 2017 08:48 AM
I decided to try a Maurits van Rees and live blog a conference talk.
Talk by Timo Stollenwerk on Building Bridges - The Headless Future of Plone



Plone's headless future

Working on what we call headless these days started in 2014
You already heard a part of this from the Keynote (about Pastanaga UI etc..) on the first day so I won't repeat that.
My ultimate goal is to bring the vision to reality.

A few observations


  • Mobile is overtaking Desktop (Plone is mobile ready but Pastanaga aims to have the best experience on every device)
  • Open Source is Mainstream (Plone is different, today large open source projects are coming from large players like Facebook and Google, this helps to make open source more mainstream). Github looked at contributions last year and visual studio code was the project that had the most contributions... Microsoft!!)
  • Javascript is taking over (Javascript is becoming more important, if you are a web developer in 2017 you have to learn modern Javascript)
  • The Web is everywhere (I visited my Uncle who is a Doctor, 5 years ago and noted that he was using a web app on his desktop for viewing scans of the body)

In recent studies they discovered that swift is losing popularity because web technologies are taking over. The web is coming back with technologies like Electron (Desktop) and Cordova (mobile)



Isn't it a great time to be a Web, Javascript, Open Source developer in 2017?

We're hearing that the CMS market is dead

If we see it in other sectors we say it is more efficient but when it happens to us we don't want to transform ourselves for the better. I think we are living in exciting times...

If JS is so great why don't we just go with it and build a CMS with Javascript?

Why do we keep using Python and Plone?

  • I love Python (wasn't my first language, but the first one I loved, I still miss Python with every line of Javascript I write). I can live with Javascript for the tooling and the community but would prefer to keep Python. I can't imagine using Node on the backend because I think Python is doing a way better job on the backend.
  • Plone the community. In the last year I've been to Jenkins, CI and testing conferences but there's no place like Plone. 
I went to a JS conference alone. Usually when you go to a conference alone you need to make an effort to talk to persons. Then I went to the sprint but out of 1000 persons there were only 20 or 30 persons at the Sprint. When I speak to Python conference attendees they ask me, how do you get people to come and even pay for a flight to Plone conferences, it's like magic!


  • Plone the Software is still unique (permissions, traversal, workflows)
  • Plone the CMS (as Eric says, Plone is doing Breadcrumbs since 2001) Go out and try all the Javascript CMSes, they all have awful user interfaces, they have nice libraries and everything you can imagine but lack the basic functionality of a CMS. I couldn't just jump and move to another system because I'd only have half or 10% of the current functionailty I have now.  We don't want to become Grandpas and isolate the new JS communities who have lots of energy


What do we have now?

Stabilising JS frameworks, it's not too hard to switch between VueJS, React, Angular. Which one you use depends on if you like a library vs a framework. How do we handle this? We want to give our clients something that can be supported over the next 5 to 10 years. That's a lot of time. If you look at Plone we are able to provide that. How do you handle that? The answer is plone.restapi (restful hypermedia for Plone).

Our idea with plone.restapi is to use it as a bridge. Stability on the backend with flexibility on the frontend. In two or three years the JS ecosystem will change further.

Status of plone.restapi

It is stable and used for 3 years, it is used in production by several companies. We consider plonerest.api to be feature complete.
I asked Eric what is plonerest.api missing, he said "nothing".

I'm just lazy about releasing a 1.0 release.

We can get back to our vision with plone.restapi being stable.

So we can get back to our vision of bridging...? One of our ideas about building with Plone 5

You can use React in the core today, if you want to go with a full framework like Angular you can too. We have 3 branches and the plonerest.api allows us to build bridges between
standard plone and the other branches.

How do we make this happen?

Regarding stories... Victor sometime says to me "Should we really say that, going on stage and telling people about these things? They will expect it to happen."

So who here would like to have Pastanaga UI today and use it. With projects in the Plone community we've building like that...



when we should be building like this..




I believe that if we want to have that we need to start with the Minimal viable product. Something, not just for users but for companies that can give them value right away. I want that skateboard.

What do we need to get that skateboard?

  • Login
  • Content Editing
  • Image upload

That might sound easy but it is not. We're aiming for making Plone stand out, we want people to be like "wow, that's the greatest editing experience I've ever had". I want us to iterate over that and focus on that user story.
I want to make image upload really easy. One thing we need to solve on a technical level, we currently have created a Medium-like editor but we don't have image scaling (something we take for granted in Plone). I want us to have the ability to add the image and have scaling done "magically".

There are things that are essential for an MVP


  • Performance
  • Image uploading
  • SEO


If we don't have this, users will abandon pages. Modern page builders focus on this (e.g. gatsby js is a modern page builder built in ReactJS). If you want to compete we have to provide users with a great out of the box performance. We'll need to use all the tooling, webpack etc...
We will also need server-side rendering. We saw that, without server-side rendering at kitconcept we can't do good SEO.

Visit https://github.com/plone/pastanaga#minimal-viable-product for details on our projections of a MVP.

Next

We have an open space at the conference and there will be sprints.
Real world projects (if you have any projects and want to use plone.restapi, angular sdk, plone.react, please talk to us)
At kticoncept we have a few projects where will do that
Sponsorship (we may be able to do something on that front)

Summary

We have a stable platform in plone.restapi for building bridges
I think Plone's future is bright is we combine our knowledge and experience with the new things
Pastanaga UI is really greate
I hope we can provide you with a roadmap
The great thing about the Plone community is tha tyou start with an idea and...
let's get together and do Plone magic together!

Plone Digital Experience Conference 2017 is live, from Barcelona!

Posted by PLONE.ORG on October 18, 2017 09:10 AM

Organizers Victor Fernandez de Alba and Ramon Navarro Bosch warmly welcomed 180 participants to the Plone Conference 2017 in their home city of Barcelona, the origin of so many parts of Plone, not the least of which is the Barceloneta theme that comes with Plone 5.

Sebastià Villa, president-delegate for Information Technology at Universitat Politècnica de Catalunya (UPC), also welcomed the conference to the beautiful venue on its campus, and saluted Plone for its contribution to the UPC, while challenging the community to continue its innovation in the direction of enhanced user experience.

In the first keynote of the day, Plone release manager Eric Steele spoke to the natural process of renewal that the Plone community has already gone through in its transition from the original generation of contributors to us, the second generation. Timo Stollenwerk laid out his vision of the combination of Albert Casado's Pastanaga user interface design and Plone's REST API, forming a new direction for the future of Plone that opens it to a much larger world of modern JavaScript front end developers.

The two conference training days have been an overwhelming success, with 90 attendees learning new skills and trying new technologies at our hands-on workshops and classes.

The conference schedule continues with today's and two more days' full slate of talks, keynotes, open spaces, lightning talks, and two days of sprints.

The annual Plone conference is an open, inviting, inclusive event hosted in cities around the world. 

For more information about the conference and the ongoing schedule of activities, see the conference site at 2017.ploneconf.org

ramon and victor.jpg

registration 2017.jpg

Conference talk schedule is live!

Posted by PLONE.ORG on October 07, 2017 06:31 PM

See the conference schedule for the full week of Oct. 16-22, 2017: 

  • training classes
  • keynotes, talks, lightning talks, open spaces
  • Plone Foundation annual general meeting
  • dinner party
  • sprints

https://2017.ploneconf.org/schedule/

Nominations Open for Plone Foundation Board of Directors

Posted by PLONE.ORG on September 09, 2017 12:50 AM

If you have an interest in helping the governance of Plone, and particularly the energy and time to pitch in, please consider nominating yourself to serve on the Plone Foundation board of directors for 2017-2018.

About Board Membership

The Plone Foundation is a not-for-profit, public-benefit corporation with the mission to "promote and protect Plone". That has meant that the board is involved in:

  • protecting the trademark, copyrights and other intellectual property, including considering licensing and usage issues;
  • hiring the release manager;
  • working with various committees, including marketing and membership;
  • handling "other stuff in the community" as needed, e.g. helping craft policy on plone.org and plone.com about commercial listings
  • but not: directing Plone development. The board facilitates, but does not direct, the development of Plone itself.

While there's lots of work that happens online, much of the critical business of the board is conducted during video meetings every two weeks — typically, board meetings last about an hour to 90 minutes though occasionally they can run over to handle time-critical issues.  Please consider whether this fits your schedule, since missing more than an occasional meeting severely limits the ability of the board to reach quorum and conduct business.

Historically, board meetings have been organized to occur during daytime hours in America and evening hours in Europe, currently at Thursday nights, 19.00 UTC in northern hemisphere summer and 20.00 UTC in northern hemisphere winter. That can always change with new board members.

In addition, there is a board mailing list (private), where we discuss things in addition to the meetings.

This is a working board. Be ready to regularly take on and complete responsibilities for board business.

The board writes no code and makes no development decisions. It is much more concerned with marketing, budgets, fundraising, community process and intellectual property considerations.

You do not need to be a Foundation member to serve on the board (in fact, board leadership is an excellent way to become a Foundation member). All you need is to get an active Foundation member to second your nomination.

The Plone Foundation is interested in broadening the diversity of our leadership, with regards to gender, ethnicity, and geography.

If you have questions about the nomination process, contact the board: board@plone.org

Nomination Process

  1. Log in on plone.org and go here: https://plone.org/foundation/meetings/membership/2017-membership-meeting/nominations
  2. Add a page there with your name in the title.
  3. For the body, discuss:
    • Who you are
    • Why you're interested
    • What you think you can add to the Plone Foundation
    • Most importantly, the name(s) of one or more Plone Foundation members who "second" your nomination
  4. Once ready, click "submit for publication" in the workflow drop-down menu to get a reviewer to look at your nomination.
  5. Nominations will be accepted until October 15 2017, 23.59, UTC. The election will be conducted in conjunction with the annual meeting, which will take place in Barcelona, Catalunya/Spain at the Plone Conference 2017. All active members of the Plone Foundation will be eligible to vote.

Help two Plonistas get from Jamaica to Barcelona (Catalonia) for the 2017 Plone Conference

Posted by David "Pigeonflight" Bain on September 07, 2017 01:23 AM

Jamaica to Catalonia for Plone Conference 2017

TL;DR - David Bain and Oshane Bailey are looking to attend the 2017 Plone Conference via crowd funding.

  • Sept 20, 2017 Update: We will mostly be walking so this reduces our transportation costs, we've adjusted our target to reflect this. We've extended the campaign until September 27.
  • Sept 17, 2017 Update: We're finding some cheaper fares, adjusting our target to reflect this
  • Sept 16, 2017 Update: Oshane has been offered a free room, this will lower the overall target further
  • Update: It looks like there are more cost effective accommodation options, as a result, we've further adjusted our estimates.
  • Update: We have found some cheaper flights via Google Flights so we're adjusting our estimates down by $2,000. 



  (David's the one on the right).

This is a manually managed crowdfunding tracker updated by David (no AI was harmed in the creation of this tracker)


We are trying to get from Jamaica to Catalonia for the 2017 Plone conference.  Our target is to raise a significant part of the roughly USD$7,000 USD$5000 USD$4,400 USD$3,700 USD$2,900 needed to cover airfare, accommodation etc.

How to support us

You can contribute to our travels via Paypal (see the button below), funds go to my Paypal account.

Why support us?

Support us so that we can deliver training, talks and participate in the sprints*.

Getting us there will allow David to deliver training and a talk or two, Oshane will be able to share his Google Summer of Code experiences and participate in his first face to face community sprint after the conference. Oshane worked this summer on improving the theme editor experience, here are some links with more information about what he did...
https://community.plone.org/t/gsoc-2017-improving-the-theme-editor-experience/3906
https://community.plone.org/t/thank-you-for-the-support-during-gsoc-2017/4792
We're hoping he'll be able to present his experience as a talk at the conference.

David has been an active part of the community for many years. He delivered training and two talks at the last conference and has been invited to be part of the training team at the 2017 conference as well.

Both of us are really excited to participate this year, however the cost of airfare is prohibitive.

* While Oshane will stay for the sprints, due to family commitments, David won't be able to stay for the sprints.

 Rough Breakdown of expenses

Identifying and fixing broken objects in a Plone website

Posted by PloneExpanse on September 06, 2017 05:30 PM
I’ve removed plone.app.stagingbehavior from a website because the new plone.app.iterate has the same functionality. In addition, the p.a.s package was overriding adapters that I wanted to write. Now, my problem was that I could no longer save any related items, I would get an error: Module ZPublisher.Publish, line 138, in publish Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 48, in call_object Module plone.z3cform.layout, line 66, in __call__ Module plone.

Tips For Writing Documentation

Posted by TestTheDocs on September 04, 2017 10:45 AM
Prologue Being busy now for more than month improving the documentation for the Plone trainings with lots of interesting and eye opening moments it is time for some words of wisdom :) First of all I would like to say thank you for all people who took time to contribute you are awesome ! As I said already before, writing documentation is not easy. Besides the knowledge about the topic you write about, you need to know how to reach your audience (tone of voice), how to structure and how to write your docs in a appealing way (more on that later).

Create Plone Training Documentation With Ease

Posted by TestTheDocs on August 21, 2017 10:12 AM
Creating Documentation Is Hard Writing documentation is not easy !. There is much more involved than just writing. You have to think about the tone, the audience, the structure of the docs and much more. Making It A Bit Easier To make this process a bit easier and with the hope to get the Plone Training Documentation better organized and unified we are happy to announce the first release of Cookiecutter Templates For Plone Training.

Announcing the Plone Conference 2018 selection process

Posted by PLONE.ORG on August 14, 2017 01:46 PM

With Plone Conference 2017 drawing near, it is time to begin planning for our next conference in 2018.The annual Plone Conference brings together users, integrators, developers, designers, and other interested folk from throughout the world for a week of training, talks, and sprinting. Plone conferences are also an expression of community spirit: they are organized by a company, user group, or other entity with ties to and a history with the Plone community and are in essence not-for-profit events.

The Plone Foundation is soliciting proposals to host the 2018 Plone Conference. The selection process this year begins in time to allow for final selection of the conference venue during this year's Conference. The extended timeline allows groups and organizations interested in hosting the 2018 Plone Conference (or beyond) to work with the Barcelona team for hands on experience during this year's conference.

Let's revisit where we've been so we can determine where we might want to go: we've traveled the world from New Orleans, Louisiana, USA for the first Plone Conference to:

  • Vienna, Austria
  • Seattle, Washington, USA
  • Naples, Italy
  • Washington, D.C., USA
  • Budapest, Hungary
  • Bristol, UK
  • San Francisco, CA, USA
  • Arnhem, Netherlands
  • Brasilia, Brazil
  • Bucharest, Romania
  • Boston, MA, USA

and this year to Barcelona, the capital of Catalunya in Spain. But, there are many places yet to explore! If you have a place in mind, don't be shy: submit a proposal!

The Plone Foundation will accept proposals beginning September 1 through September 28, 2017.

The Foundation Board of Directors will review proposals and open those that are viable for voting by the Foundation membership between October 9–13, 2017. The winning proposal will be announced at the end of Plone Conference 2017 in Barcelona.

Everything you need to know to submit a proposal, including the full schedule for the process and in-depth requirements for hosting, is outlined in the official Plone Conference 2018: Call for Proposals.

On behalf of the entire Plone community, we look forward to your conference proposals!

ttd-textlint released

Posted by TestTheDocs on August 14, 2017 11:38 AM
0.0.1 Released ! We are happy to announce the first release of ttd-textlint. The Plone Conference 2017 is getting closer. You should attend ! Even if you are not using Plone there are plenty interesting topics and a truly awesome community ! As you may know, there is also training included, yes that is right, no extra costs for trainings as the fees are already included into the ticket price !

Why you had problems figuring out Plone (the webinar)

Posted by David "Pigeonflight" Bain on August 03, 2017 10:32 PM


Presenting... Why you had problems figuring out Plone (the webinar). Okay, that's not the actual name of the webinar. Instead, we went with the more descriptive but slightly less clever Plone for Newbies - The Big Picture.

The Big Picture is about understanding the model.

If you're a developer about to begin your journey of Plone development, The Big Picture aims to fill out your understanding of how the pieces of Plone fit together. Thinking of it as a purpose built system lays a strong foundation for success.

Only smart persons use Plone

I've heard someone suggest that you have to be really smart to use Plone (implying that it is hard to use). I call it the "this helicopter is harder to use than my bicycle" problem. Every time I benefit from Plone's link integrity support, flexible access control model or use cut and paste to move content around I'm glad I'm not using a "bicycle". I like to point out Plone's comprehensive suite of tools which you'll be glad exist when you need them. For developers, once you accept that you're looking at a "helicopter" you need to spend a little time "understanding the model". As you understand the purpose of the major controls you'll find it easier to use the system to solve problems.

Why Plone? 

Now is the right time, with an increase of cybersecurity related issues, organizations should be looking to adopt secure platforms. I've been using Plone for more than 15 years, and I can confidently recommend it as a secure platform. In case you took your eye of Plone for a few years, now is a great time to give it a second look, it has kept up with modern development practices and remains an excellent choice for your content management needs.

Why a webinar? 

My target audience isn't in one geographic location, the most effective way to reach them is a virtual medium and webinars provide a well-known, tried and tested approach.
Additionally, I have run one or two webinars before, if you count online training courses. Of course, it is different when you are doing more than just showing up, reaching out to a "less captive" audience and convincing them to commit 90 minutes to a webinar. So this is new ground for me. I am learning a lot from this experience and have had a fleeting thought, maybe I'll take all this webinar and Plone stuff and do the "meta" thing, build a webinar management tool on top of Plone. You never know.




Restore missing blobs from blob cache

Posted by PloneExpanse on August 03, 2017 01:40 PM
I had a curious case of missing-but-present blobs in an old Plone service, configured with a Zeo server and 2 Zope instances. The root of the problem (I think) was that the blob folder configuration was broken: the Zope client instances were configured with shared blobs to “off”, but they were really sharing the same caching folder. In the end, the blobs were loaded by the Zope services and everything appeared to be working, but when I’ve tried to move the blobstorage folder to a new machine, I ended up with missing blobs.

Submit your talks, take classes, and see the full keynote lineup: Plone Digital Experience Conference 2017 in Barcelona

Posted by PLONE.ORG on August 02, 2017 06:40 PM

Hear ye, hear ye!

We are accepting talk proposals now: submit your talk for any of the three tracks (Plone, Python web frameworks, modern JavaScript).

See our complete lineup of keynotes

  • Denys Mishunov (all-round frontend developer)
  • Éric Bréhault (Plone Framework Team member / Makina Corpus)
  • Eric Steele (Plone Release Manager / Salesforce)
  • A. Jesse Jiryu Davis (Staff Engineer at MongoDB)
  • Simona Cotin (Cloud Developer Advocate for Microsoft)
  • Mark Pieszak (Angular Universal team member)

Check out the list of training classes that you can take FOR FREE (included in your conference ticket) and use the training signup form to reserve your spot.

Buy your tickets now before all the early bird discounts are gone!

Image credit: https://www.flickr.com/photos/x-ray_delta_one/8378622137

Report from Midsummer Sprint 2017

Posted by PLONE.ORG on July 17, 2017 01:55 PM

There were 18 participants at Midsummer Sprint in total. The participants included, of course, the organizers, the local development team for Plone based and related services, a GSOC student working on a Plone Foundation mentored topic, a professional UX designer and many well known names from the Plone community.

While the sprint was unable to fix as many content editing related issues as was hoped for, it definitely managed to get the best out of its participants:

The local developers submitted their Plone contributor agreements to be able to push their patches upstream, they learned how those patches were made, tested and reviewed in practice, and they got to know many experienced Plone developers in person. They researched and fixed many Plone 5 issues with PloneFormGen.

Maik Derstappen did spectacular work at the sprint by cleaning up and fixing the code base for a major security feature of Plone called safe HTML transform.

Peter Holzer continued on earlier work by Jens Klein and introduced an optional Show Toolbar permission to control when the full toolbar is shown, and an alternative member tools dropdown menu to show a minimal set of required actions (including logout) for logged-in users when the real toolbar is not shown.

Peter Holzer and Asko Soukka redesigned the Translate menu of Plone Multilingual support to be much simpler and more consistent with the other menus in Plone's editing toolbar.

Stephan Klinger championed to implement the PLIP for adding the missing redirection management UI for Plone, based on the existing (but lacking) RedirectionTool add-on.

Eric Steele and Philip Bauer completed the long-awaited refactoring of Plone login forms.

Sven Strack, Paul Roeland and Alexander Loechel worked on better tools for building and testing the documentation, a better theme for the upcoming Plone 5.1 documentation, better readability of the documentation on mobile devices, improved documentation on how to contribute to the Plone documentation and more. In addition, Paul fixed a few reported accessibility issues in Plone 5.

Other interesting developments at the sprint included Maik implementing sub-templates into bobtemplates.plone, Alexander adding tox-based test setup into bobtemplates.plone and Jussi Talaskivi enhancing plone.docker.

Albert Casado completed the design for Pastanaga UI components and composed a visual style guide to help with the reference implementation and later adaptations. Victor Fernández de Alba and Timo Stollenwerk supplemented Albert's work by bootstrapping a ReactJS-based reference implementation of the Pastanaga design.

For more details, see the final report on Plone Midsummer Spring 2017.

Henry Alpha 1 Released

Posted by TestTheDocs on July 16, 2017 04:55 PM
First Alpha We’re happy to announce the first alpha release of henry ! Get it from GitHub ! Make sure to read the release notes ! Status At the Plone Midsummer Sprint we gave a talk about the state and future of the docs. During this talk we introduced henry to the Plone Community. Throughout the sprint week, participants started already using henry for building the docs. Even henry is still alpha and has some rough edges it works already.

Report from Plone Open Garden 2017

Posted by PLONE.ORG on July 05, 2017 07:50 PM

Report authors: Christine Baumgartner, Jens Klein, Fred Van Dijk

Plone Roadmap and Future Directions

Plone community members from around the world once again met for the 11th Plone Open Garden (PLOG) at the Hotel Mediterraneo in beautiful Sorrento, Italy, from April 18 to April  22, 2017.

The aim of the gathering near Naples was

  • to discuss and refine the roadmap for Plone 6
  • to continue work on the new Plone Headless CMS initiative
  • to decide on a new way of integrating Plone with its increasingly important JavaScript components.

PLOG’s new stewards

“PLOG is a strategic sprint for the Plone community,” explained Jens Klein, a returning attendee from Austria and member of the Plone Framework Team.

For the first time since PLOG's founding by Abstract Technology in 2007, the organization of the sprint was handed off to the general Plone community. Thanks to the gracious efforts of Abstract’s Vicente Barone, Rosario Savarese, and Maurizio Delmonte, the transfer went off without a hitch, continuing the tried-and-true formula that has worked for years.

“Networking, getting a feeling of current directions of the project, and identifying integrators' operational needs” are the reasons why Christian Theune traveled to PLOG from Germany to participate in discussions.

New opportunities with the Plone Headless CMS

The web’s front end, rendered by JavaScript running in the browser, has been evolving away from server-generated HTML and toward pure client-based rendering, which provides faster, richer user experiences, especially on mobile devices. The fast pace of JavaScript framework development has made it difficult for developers to choose a framework that will be around for the long haul.

In contrast, Plone as a back end storage system is stable, secure, and scalable, and comes with a complete, time-tested set of data management, workflow, and authorization functions.

With the Headless CMS initiative, the Plone community has found a way to combine Plone’s back end strengths with the richness of the rapidly changing JavaScript landscape.

The Plone community’s decision in 2014 to proceed with the adoption of a full API and accompanying REST API has come to fruition by making possible the decoupling of the rapidly changing JavaScript front end from Plone's stable back end.  

Plone as a headless CMS, in turn, offers the JavaScript world a mature and flexible back end on which to build its web and mobile applications.

In a video presentation at PLOG, Ramon Navarro-Bosch from Barcelona and Nathan Van Gheem from Green Bay, USA, shared their experiences experimenting in depth with the headless CMS approach using an Angular-based client with Plone’s REST API.

Eric Bréhault from Toulouse, France, travelled to Sorrento to join the work being done on the headless CMS initiative. “I came to PLOG because I wanted to present its current status, discuss strategy and see how it could fit in the Plone roadmap.”

Timo Stollenwerk from Bonn, Germany, presented by video the current status of the Plone REST API design, which follows best practices and the latest standards.

Maurits van Rees came to PLOG from Rotterdam seeking “Confirmation that we are not doing overly weird things that are just the pipe dream of individuals, but doing things that are actually possible for us, as a community.”

New UI/UX concept: Pastanaga

The draft of a fresh mobile first UI/UX concept created by Albert Casado was presented by Víctor Fernández of Barcelona. The concept is to make authoring content easy by focusing on the tools needed. The concept UI is named Pastanaga, meaning "carrot" in Catalan. While the concept is still in its initial stages, the community hopes to refine it and user it at future sprints.

“Albert is doing a terrific job!” said Gil Forcada Codinachs, Plone's continuous integration guru from Berlin.

Strategies for better JavaScript/CSS integration

Working with JavaScript and CSS in Plone 5 continues to require updates  because of the high rate of change in the JavaScript front end world. At PLOG, sprinters came up with an improved way to integrate JavaScript with Plone that decouples JavaScript from Python packages. The community plans to use the same tools used by the JavaScript world, such as Node packages (npm) and webpack configurations. 

The Roadmap: the future of Plone

The sprinters included ideas from the headless CMS, plone.restapi, and JavaScript framework integration into the Plone roadmap and developed a timeline to release these and other planned innovations.

“The most important outcome is the roadmap we agreed on,” explained Eric Bréhault.

According to Jens Klein, “The roadmap shows what we want to have, in which Plone version. We plan to not have a major Plone 6 release immediately, but wait for user-visible changes that will come with the new Pastanaga user interface”.

In memoriam: Jean Ferri

Posted by PLONE.ORG on June 30, 2017 07:36 PM

Jean Ferri, Plone Foundation member, recipient of the 2012 Plone Award, and one of the organizers of Plone Conference 2013, passed away yesterday, June 29, 2017.

He was the founder of TcheZope.org, the first site about Zope and Plone in Portuguese, leader of the Portal Modelo project from Interlegis, the Plone portal for the legislative houses of Brazil, and the founder of the PloneGov.Br and Associação Python Brasil (The Brazilian Python Association).

He was involved in the organisation of PyCon Brasil 2006, the Plone Conference 2013, and gave talks about Plone in dozens of events in South America.

Most importantly, he was an open source software hero, an idealist who helped and inspired a whole generation of Plone developers in South America.

Jean was a wonderful human being. On behalf of the entire Plone community, the Plone Foundation Board gives its condolences to Rafahela and Rodrigo in this sorrowful moment and wishes them resilience and spirit.