The template include size is exceeded.

<!-- 
NewPP limit report
Parsed by deployment‐mediawiki04
Cached time: 20170818152931
Cache expiry: 3600
Dynamic content: true
CPU time usage: 10.684 seconds
Real time usage: 12.470 seconds
Preprocessor visited node count: 486033/1000000
Preprocessor generated node count: 0/1500000
Post‐expand include size: 2097152/2097152 bytes
Template argument size: 1771720/2097152 bytes
Highest expansion depth: 26/40
Expensive parser function count: 8/500
Lua time usage: 0.740/10.000 seconds
Lua memory usage: 11.89 MB/50 MB
-->
<!--
Transclusion expansion time report (%,ms,calls,template)
100.00% 11113.651      1 -total
 77.11% 8569.383      1 Template:Reflist
 52.10% 5790.588    403 Template:Cite_news
 52.07% 5786.480    410 Template:Citation/core
 21.99% 2444.354    933 Template:Citation/showdate
 16.32% 1813.776    485 Template:Accessdate
 15.84% 1760.239    153 Template:Cite_web
 15.70% 1745.072    666 Template:Citation/showdateCore
 14.23% 1581.604    485 Template:Accessdate/core
 10.25% 1139.677   1151 Template:ISO_dateJA
-->

Dear Release engineering team, do we know how to increase this size? If we are purposely keeping it low what do you recommend for maintaining a copy of a page on production in the beta cluster environment?

Yes, it's controlled through a combination of several settings: final page size, expansion depths, etc etc etc. As far as I know, these are basically all kept to default core values except $wgMaxGeneratedPPNodeCount. They're not purposefully low, and definitely are the same between beta & production--curious why production doesn't fail in the same manner.... I'd recommend having an article with fewer references 🤔

I'm hesitant to just raise the values blindly, needs more research first from someone who really understands this better than I (or anyone else in RelEng)

@Peter it might be worth changing the article we test on if this cannot be resolved. We're not able to use the beta cluster as a test for production while this is a problem.

An alternative solution might be to find a way to generate the same HTML without templates. @Tgr is that possible?

Do you mean something like https://en.wikipedia.org/w/index.php?title=Barack_Obama&action=raw&templates=expand ?

@Tgr that works perfectly. I've added an inline comment to tell us to use that api to get the HTML. There are a few inconsistencies (see Cite web links) but it will probably help things. I'll let the webpagetest job run a few more times before confirming this addresses our need to detect performance regressions on the beta cluster before they go out.

Using expanded templates doesn't seem to help completely. There's still approx 23kb difference between the two pages which is unfortunate.

Do you have a diff? Probably there are subtle differences in how transcluded content inside extension tags (such as <ref>) is handled, but that sholdn't be that large.

Diff is too messy to be useful, but the main visual differences are red links and certain templates that failed to render (see the Template:Cite news in references)

I think the only possible solutions here are:

• Use a different article (a pain in the ass and we'll lose historic data)
• Increase the template include size on beta cluster

can anyone from the parsing team shed light on the root cause here?

Probably a difference in citation templates? Those do not get expanded even with templates=expand, due to a parser bug (T48115). That shouldn't be a problem in that pre-expanding everything that the parser is willing to pre-expand will still reduce the resource consumption.

If that doesn't work, given that this is beta, you can always just parse the wikitext manually from a php shell, with custom limits.

The expansion limits are same on beta and production, so unless some template/module is more complex on beta enwiki than on real one (seems unlikely) this is more likely to be some kind of timeout / memory overrun than parser expansion limits.