Mel Choyce’s Recap on WordPress 4.9

Whereas my recap post about WordPress 4.9 focused mostly on the new features and enhancements, my co-release lead Mel Choyce just published a great post that gets into more of the process aspects of the release, including the key contributors:

WordPress 4.9 Released

I think that 4.9 went really well by having essentially three co-release leads: Mel Choyce leading design, myself leading the development, and Jeff Paul leading project management as deputy release lead.

WordPress 4.9

For the past few months I’ve been co-leading the WordPress 4.9 release with Mel Choyce (and Jeff Paul too). I’m really excited about all the new features and enhancements we’ve built which will now be rolling out across 29% of the Internet(!).

WordPress 4.9 “Tipton”

In particular I’m excited about the powerful new capabilities being added to the Customizer. Now you can draft changes to come back to later while also scheduling them to be published at a future date.  The Customizer also now features autosave revisions, changeset post locking, customization drafts, and more. These changes were a long time in coming. For more than two years we’ve been working on these features in the context of the Customize Snapshots feature plugin where we’ve developed the infrastructure and UI for changesets in the course of service delivery for our clients at XWP. In WordPress 4.7 the changeset infrastructure from the plugin was introduced to core, and now in 4.9 the UI has also been introduced, after many design iterations.

I’m also particularly excited about the improvements to the Customizer JS API in 4.9. It is now much more pleasant to develop applications in the Customizer using JavaScript, where the JS API is much more of a first-class citizen on par with its more popular PHP API. Since the Customizer is WordPress’s first single-page application, JavaScript has always been key. However, the PHP API has been the primary way that developers would interact with the Customizer, where all of the PHP API calls get translated into JS API calls when the Customizer loads. Now with 4.9 developers should feel much more inclined to target the JS API directly without having to use with the PHP API abstraction. Ultimately, if you want to make scalable applications in the Customizer, you must use the JS API. I’m going to be talking about building with JavaScript in the Customizer at WordCamp US this year (talk now available).

There are several other things I’m really happy about in 4.9, including the integration of CodeMirror into the code editing interfaces (Additional CSS, Custom HTML widget, theme/plugin editors) and the many improvements to widgets, including adding media/embeds to the Text widget and a new gallery widget.

I presented the new highlights in 4.9 (and 4.8) during the State of the Word:

Presenting WordPress 4.8 & 4.9 at State of the Word 2017

I’ve written about all of these improvements in detail over on Make/Core. Since I don’t post much here on my personal blog, here’s a roundup of the 4.9 posts I’ve published there:

New Features and Enhancements with Customizer Changesets in 4.9

Improvements to the Customize JS API in 4.9

Widget Improvements in WordPress 4.9

Code Editing Improvements in WordPress 4.9

Introducing the Gallery widget

Thanks to XWP for sponsoring my time to be part of this release.

WordPress 4.8

We did a ton of work on widgets in the WordPress 4.8 release. Prior to this release there hadn’t been any new widgets introduced for a very long time. Now there are media widgets for images, video, and audio. Additionally, the Text widget finally gets the TinyMCE visual editor just like the post editor has had forever. In 4.8.1 we introduced a dedicated Custom HTML widget for storing arbitrary markup, which the Text widget was formerly used for. Many of these widget changes incorporated ideas from XWP’s JS Widgets plugin, as prior to 4.8 widgets in core lacked any JS-driven functionality at all.

The widget improvements aim to mirror the development of blocks in Gutenberg. If there is a block for something, there should also be a widget for it.The more that block-like widgets are available today, the more users will see a consistency between post content and widget sidebars. After Gutenberg is merged into core, we’ll be working on migrating widgets over to use blocks instead. At that point we’ll be able to abandon many of the hoops we have to go through to make widgets JS-driven when they are fundamentally PHP beasts.

Here are a few of my posts from WordPress 4.8 & 4.8.1 releases on Make/Core:

Media Widgets for Images, Video, and Audio

Addition of TinyMCE to the Text Widget

Fixes to Text widget and introduction of Custom HTML widget in 4.8.1

Customizer Features for WordPress 4.3 Kickoff

Here are the features I have suggested/proposed during the WordPress 4.3 kickoff:

Partial Refresh

This greatly improves performance of previewing changes in the Customizer for non-postMessage transport settings (JS-applied changes) by just refreshing the area of the page that has been changed. As such it eliminates some of the need to do postMessage in the first place, while also reducing the amount of duplicated logic that would have to be implemented in JS to mirror what is being done in PHP. This resurrects some code from the old Widget Customizer feature plugin developed for 3.9. Writeup and feature plugin are available.

Transactions

A low-level re-architecture of the Customizer plumbing that has a lot of side benefits and bugfixes, introducing some exciting possibilities for future feature plugins like scheduled settings, setting revisions, and drafted/pending settings. Partial Refresh is a dependency for this. Pull request available, but needs refresh. See proposal.

Concurrency/Locking

This is an important one for a client project I’m involved with, and so I’m having to prioritize it. I’m working on a client site that will have many users in the Customizer at a time, and given the way the Customizer is currently implemented (as with most areas of WP), there is no concurrency/locking support. So I’m working on adding locking at the control/setting level. See #31436.

(Cross-posted in comment.)

WordCamp Sydney talk: Customize all the things!

Today at WordCamp Sydney, I speak on The Customizer:

The Customizer is one of the least known yet most powerful features of WordPress. We need to get past thinking of it as the “Theme Customizer” which only good for tweaking colors and a limited number of settings. No, the Customizer provides a framework for live-previewing any change to WordPress.

In WordPress 3.9 I led development of incorporating Widget management to the Customizer, and I’d like to get on a soapbox to get people excited about developing their own custom controls for the Customizer.

Here’s my presentation:

I’m also happy that we at XWP (X-Team WP) and Stream are among the sponsors of the WordCamp:

Re: Edit WordPress Posts and Postmeta in the Customizer

I couldn’t reply to Jeff’s comment on Sarah’s WP Tavern post because of a Jetpack problem, so I’m posting it here:

With the Front-end Editor using the Customizer as a framework, I wonder if the interface will look similar to this experiment? I hope not.

This plugin is really a prototype to demonstrate how the Customizer can be used to preview changes to posts/postmeta at an architectural level. Hardly any consideration has been given to UI, which Avryl has been doing a great job with in her Front-end editor.

Ultimately, I’m interested to see how the Front-end Editor comes into play when previewing a post or writing a draft post. Will we be able to use it to write a post or will the editor be refrained from use until after a post/page is published?

You can use this Customize Posts plugin to edit drafts as well. Just create a new post post, save a draft, and then click Preview. You can then click Customize in the admin bar to then access that post from the Customizer and you can make changes, including transitioning the post draft to a published post. This relates to a previous prototype I worked on in the Customizer Everywhere plugin, which replaces the “Preview” button with a “Preview & Customize” button.

In this case and as I’ve seen mentioned already, editing should be done inline and not with boxes popping up around the content.

In terms of a prototype which demonstrates how inline editing can be added to the Customizer, see my other prototype: Customize Inline Editing.

Failures related to the WordPress APC Object Cache plugin

On a site for a newer X-Team client, APC was previously chosen as the object cache plugin. This has caused some headaches due to APC not being available while in CLI mode, rendering WP-CLI mostly broken: any commands which change parts of the database which get object-cached won’t appear until the cache gets invalidated, which may not happen anytime soon. For example, plugin (de)activation is broken in WP-CLI with APC Object Cache, as the manifest of activated plugins is cached.

Today I encountered a more serious problem with APC Object Cache.

I was making some widget changes, and I saw that my changes weren’t being reflected on the frontend. Upon reloading the widgets admin page, I also noticed that my changes didn’t seem to get saved. But if I ran wp option get widget_text, I could see my changes had been written to the database (as, of course, WP-CLI here was bypassing the APC object cache). So for some reason, APC was refusing to invalidate widget caches. Doing some more digging, I was shocked to see that the the Text widgets used in the sidebar, had multi-widget instance numbers surpassing 560,000,000. That’s right, 560 million.  The output of wp option get widget_text included:

array (
  /*...*/
  526001242 =>
  array (
    'title' => 'Lorem Ipsum',
    'text' => 'Hello World',
    'filter' => false,
  ),
  '_multiwidget' => 1,
)

Some plugin must have at some time bumped that widget instance number up, because there’s no way there have been millions of Text widgets used on the site. Only a couple dozen Text widget instances are currently present.

In any case, this incredibly-sparse PHP array seemed like a good candidate for what was causing APC cache to fail. I swapped out APC for Memcached Object Cache, and the problem went away. While the Memcached plugin stores arrays as serialized strings, just as they get stored in the options table, the APC plugin is apc_store’ing the array wrapped in an ArrayObject instance:

if ( is_array( $data ) )
    $store_data = new ArrayObject( $data );

While I couldn’t reproduce this problem on VVV or on the site’s staging environment, for some reason on production this array object failed to get stored by APC.

I’ve never had any issues with Memcached Object Cache, so I’m glad I made the switch. Not only did it fix this strange widgets issue, but it’s great to be able to once again rely on WP-CLI!

Easily turn any webcam image into video on an OS X Dashboard widget

I have a few webcams on my network which I can access via URLs like http://198.162.1.30:8080/cam.jpg. Here’s a handy little bookmarklet (pagelet?) which allows you to turn any such webcam URL into live video (here, 1 fps):

data:text/html,<img src="http://198.162.1.30:8080/shot.jpg" onload="var img = this; setTimeout( function(){ img.src = img.src; }, 1000 )">

Paste this  data:  URL into Safari and then turn it into a Dashboard widget via File > Open in Dashboard. Now you have a video Dashboard widget for your webcam! (Of course you could create a standalone HTML page and load it instead, but data: URLs are cooler and easier!)

WARNING: This could get interpreted as a Denial of Service (DOS) attack if you point this at a URL on a public web server. So it’s best to only do this for cameras on your local network.

Of course, if your camera supports MJPEG aka server-push (multipart/x-mixed-replace), you should just reference that MJPEG URL directly without any such code above.

Regularly cleaning up /tmp on a DreamHost VPS

I ran into a problem recently where I was no longer able to upload a file via PHP. I checked the server error log, and I saw entries like:

ModSecurity: Input filter: Failed writing X bytes to temporary file

Looking at my /tmp directory it contained 128MB of data. Apparently ModSecurity tries to prevent the filesystem from being maliciously filled up.

If you try to manually clean up just with a rm -rf /tmp/*, it will fail because the files are owned by dhapache and are not writable by anyone else. But, with DreamHost VPS, you have the ability to add admin users (aka sudoers). As an admin user, you can then set up a cron to automatically clear out the /tmp directory of old files:

su myadminuser
sudo crontab -e

Then in the editor which opens, add this line:

0 0 * * * find /tmp -mtime +5 -exec rm -rf {} \;

This will delete all files under /tmp which haven’t been modified in 5 days; it will happen every day at midnight.

Re: Sharing Sidebars Across a Multisite Network

In a post yesterday at WebDevStudios about “Sharing Sidebars Across a Multisite Network”, I was reminded of a workaround that I had to put into the Widget Customizer plugin. There is this unfortunate condition inside of the core wp_get_sidebars_widgets() function:

// If loading from front page, consult $_wp_sidebars_widgets rather than options
// to see if wp_convert_widget_settings() has made manipulations in memory.
if ( !is_admin() ) {
    if ( empty($_wp_sidebars_widgets) )
        $_wp_sidebars_widgets = get_option('sidebars_widgets', array());
    $sidebars_widgets = $_wp_sidebars_widgets;
} else {
    $sidebars_widgets = get_option('sidebars_widgets', array());
}

So you can see here that if wp_get_sidebars_widgets() gets called early (and it does) then the initial value returned will persist even if the underlying sidebars_widgets option changes (or is filtered); after this function is called on the site frontend, the returned value will persist in that private global variable $_wp_sidebars_widgets.

This was a problem for Widget Customizer because it needed to be able to override the sidebars’ widgets on the fly within the customizer preview. Fortunately, the return value of wp_get_sidebars_widgets() is filterable via the sidebars_widgets hook, and so this is what we did to get around the above behavior:

add_filter( 'sidebars_widgets', function ( $sidebars_widgets ) {
    $sidebars_widgets = get_option( 'sidebars_widgets' );
    unset( $sidebars_widgets['array_version'] );
    return $sidebars_widgets;
} );

Would this workaround also allow switch_to_blog() to be able to render sidebars from other blogs on the network?