Lua scripting/status

Last update on: 2013-04-monthly

2011-08-31

Volunteer Victor Vasiliev worked on a MediaWiki extension to embed scripts into pages; this was a result of discussions over the years about replacing ad-hoc template- and ParserFunctions-based logic by a more efficient and powerful solution. Tim Starling discussed the extension with Victor to become more familiar with his work, and researched other alternatives. He (Tim) wrote a PHP extension embedding a Lua interpreter, and added support for it to the existing Lua MediaWiki extension for backward compatibility.

2011-09-30

Tim Starling did some prototyping and testing on this project. He created a test harness that can be used to trial the various approaches, and made a Wikitext-to-Lua template converter to test the relative performance.

2011-10-31

This project was mostly on hold in October.

2012-01-26

We made a decision to move forward with Lua. Details coming to wikitech-l

2012-01-31

A team of Wikimedia engineers agreed on Lua as the language to implement as a production-ready replacement for MediaWiki markup-based templates. Tim Starling will lead this effort after the 1.19 deployment and Git migration.

2012-04-05

Tim Starling is currently working on the replacement of ordinary wikitext markup with Lua and says that he is halfway done, and hopes to finish in the next few weeks.

2012-04-19

Tim's work continues. The standalone engine is working and Tim is working on a standard library (it's pretty small so far). Lua can't yet call templates. There will be two ways of passing text (Tim will post to wikitech-l later about it). Parser integration features are remaining, and Tim may ask WMF's Features team for help. We'd like to give people a sandbox/console feature so people can play with it (this would be a CodeEditor feature request).

The Lua timeline is part of the site performance timeline. We aim for Scribunto and its Lua engine to start running on a small, simple test wiki in Labs in May, so other people can start to try it out. Then the aim is to get feature complete, deploy to mediawiki.org, leave it there a few months to look for technical problems and to discuss it with the community, and only later deploy to all other wikis. Per the Goals timeline, it will be early 2013 by the time of rollout to WMF sites.

To help prepare for this, Tim will lead tutorial sessions at Berlin Hackathon 2012. There will be an intro to programming in Lua (for people who have programmed in other languages before) and an intro to programming for people who have only ever made wikitext templates. In both cases, he'll show simple examples of how to convert a wikitext template into Lua. At the end, every participant will have actually converted a real MediaWiki template into Lua.

2012-04-monthly

Tim Starling started the implementation of a replacement for MediaWiki markup-based templates, using the Lua scripting language, embedded via the Scribunto extension. The current roadmap aims for a deployment to Labs in May 2012, then to mediawiki.org; full deployment to Wikimedia sites is scheduled for 2013. Tim will lead tutorial sessions at the Berlin Hackathon 2012.

2012-05-10

You can try it now at the Labs prototyping site. Patrick Reilly spoke about the Lua scripting work in a presentation on 3 May 2012 (scroll to about 22:00 or 27:00 to see it).

2012-05-monthly

Tim Starling's Scribunto extension is now available for testing in Wikimedia Labs. Tim also led tutorial sessions at the Berlin Hackathon 2012.

2012-06-18

Tim Starling led tutorial sessions in June and videos (first session, second session) are now available on Vimeo. They will be on Wikimedia Commons by mid-July. Ross Andrews is now working on documentation in the form of help/tutorial pages, especially describing the MediaWiki interface. Once that's done, Tim will promote the prototyping site on Labs more heavily, and at some point after that, we will install the Scribunto extension on mediawiki.org. Full deployment to Wikimedia sites is scheduled for 2013.

2012-06-monthly

Tim Starling led tutorial sessions in June and videos (first session, second session) are now available on Vimeo. They will be on Wikimedia Commons by mid-July. Ross Andrews is now working on documentation in the form of help/tutorial pages, especially describing the MediaWiki interface. Once that's done, Tim will promote the prototyping site on Labs more heavily, and at some point after that, we will install the Scribunto extension on mediawiki.org. Work on Lua was paused in late June to catch up on other activities. Full deployment to Wikimedia sites is scheduled for 2013.

2012-07-monthly

Tim Starling has added a debug console to test code snippets. We believe we're ready to deploy Lua to the WIkimedia cluster, starting with test2 in August, followed by mediawiki.org. We plan to let Lua incubate on mediawiki.org while we test the performance characteristics with key templates, and work out a deployment plan for larger wikis that includes community involvement.

2012-08-monthly

The Scribunto extension has been deployed to test2.wikipedia.org and www.mediawiki.org, and several editors are porting existing templates such as Cite over to Lua (see recent changes in the "Module:" namespace)

2012-09-06

Tim Starling: "I finished work on a basic profiler yesterday, and now I'm adding some time and date functions to the default environment."

2012-09-monthly

Tim Starling created a basic profiler to work with Lua code, added some time and date functions to the default environment, and fixed bugs. Experimentation continues with Lua on mediawiki.org and Lua on test2wiki.

2012-10-18

This has been on hold with many other small projects and site issues taking priority. Next up for Tim is to gather more profiling data, comparing Lua template performance to Wikitext template performance.

2012-11-20

Brad Jorsch and Chad Horohoe have joined Tim Starling on this project. Brad has built a TemplateSandbox which will help in debugging Lua scripts (and old fashioned templates, too!). Chad is working on a shared repository for scripts, and Tim has been extending the API. His latest work has been around adding multi-lingual APIs for handling things like plurals and such within Lua. We're currently seeking a volunteer product manager to help out with the rollout of this.

2012-11-monthly

Brad Jorsch and Chad Horohoe have joined Tim Starling on this project. Brad has built a template sandbox which will help in debugging both Lua scripts and regular templates. Chad is working on a shared repository for scripts, and Tim has been extending the API. His latest work has been around adding multilingual APIs for handling things like plurals within Lua. We're currently seeking a volunteer product manager to help out with the roll-out of this.

2013-01-monthly

Lua development was put on hold through the Ashburn data center migration. We've now resumed work on Lua, with Brad Jorsch and Tim Starling making more functions available in Lua that are currently already available in template parser functions.

2013-02-monthly

We deployed Lua/Scribunto to several wikis, including English Wikipedia, on February 18th. The current plan is to deploy to the remaining wikis on March 13th.

2013-03-monthly

We launched Lua scripting on all wikis, wrote about the launch's wider significance, and held IRC office hours. In March we also added frame:callParserFunction() and frame:extensionTag(), improved CPU time accounting, and allowed argument expansion to be excluded. We have patches outstanding for "text" module including unstrip functionality, as well as improved debug output. We've also made significant improvements to templates since the launch.

2013-04-monthly

Some bugs were fixed and internationalization changes merged this month; no major changes were made. The community continues to develop Lua-based templates, such as the citation templates on the English Wikipedia.