Looking for advice, tips and tricks on playing a MASSIVE 2D animation on mobile phones

Help Animation
, , , ,
1

Godot Version

v4.6.2.stable.mono.official [71f334935]

Question

This post is more of a “I need some advice on how to do a certain thing”.
I’m currently working on a mobile game that will have to run on both Android and iOS devices, and needs to be finished within a few weeks.
Without going into too much details, since most of it is under NDA, I’m faced with a challenge that I’m not entirely sure how to approach.

The game has 3 “areas”, and each area contains a massive background animation which the player can / is encouraged to explore by “pinch to zoom” and “dragging” the camera around, and each area also has 3 different stages of animation.

Now, the main challenge is that when I got ONE stage of ONE animation, I noticed that it contains a total of 239 Frames, and a single frame is 4,000 x 8,889 in resolution, and around 32MB in size. (It should run at 60 FPS so this one animation is roughly 4 to 5 seconds long)

Obviously, there’s no way I can just display this as a single Sprite2D and animate it on even most modern phones without issues, and on older phones I already confirmed that it will just refuse to load the texture into video memory.

Now I’m trying to figure out how to best approach this problem.
Here are the few ideas I had, but why I think they might not work:

Idea 1:
Instead of using PNG sequences, I’d try and convert the entire PNG sequence into a video file, and try to play that back. The issue with this approach is that I’d still like to keep the quality nice, especially when the player zoomed in. Another issue is that as far as I know, Godot can only do software playback, it cannot use hardware acceleration for videos at all.

Idea 2:
I’d try to take each PNG into smaller pieces, and have several small Sprite2Ds that I would have to keep in sync and update their frames multiple times each second. The problem with this approach is that I really don’t think this would be efficient. And I also can’t see how older devices would be able to decode PNG files fast enough. Even if I tried to implement a streaming solution where I replace the art with higher resolution versions as the player zooms in more, I don’t see it being fast enough.

All in all, I’d like to find a balance between efficiency and still keeping a high resolution version of the artwork for players to look at, even on older devides. (even if it means on older hardware, I’d have to pause the animation entirely). But I’m not even sure how to get started with this.
Thank you for any tips in advance!

2

I once worked on a project where the manager insisted that data processing needed to happen client side - with a data set that took over 15 minutes to load on the company network. This kind of feels the same.

I wish you luck in finding a solution - but will not be particularly surprised if you can’t. I think you have a blivet on your hands.

3

Are you sure you need that much of a resolution? How the zoom happens? It this just a cutscene or there’s some interaction involved?

Video would be the best approach as streaming/playback will likely be more optimized than anything you can hastily knock up on your own.

4

It’s not a cutscene, it’s a persistent animated background that the player will interact with. It has several point of interests using TouchScreenButtons and similar interactable elements, so it needs to look good even when the player zooms in quite a bit.
I can probably get away with dropping the resolution a tiny bit, but not too much.

It’s through a custom Camera2D using Pinch to zoom.

5

Hard to tell without seeing it but maybe the solution could be more on the art side than on technical side. Have a large static (or parallaxed multi-layer) background and then sprinkle smaller overlaid parts that are animated to sell the illusion that everything is animated. Keep the points of interests separated and make them visually stand out from the rest. In other words, do a clever collage where you animate much fewer pixels than it really seems.

6

There’s not much to show, it’s literally just a massive PNG sequence, nothing special. And I’m unable to get each individual layer because the artist used After Effects with several different addons to create the animation itself.

7

Yeah, it’d require some additional work on artist side to adapt it. You could just tell to the decision maker that things must be adapted to technical constraints.

Other than that, as small as you can afford video is your best bet imo.

8

So you are being set up to fail because the artist has created something that doesn’t work in the environment it is designed for. Not cool.

From a purely office politics perspective I would bring this problem up to my manager immediately.

9

It’s volunteer work and I don’t blame the artists at all, they made the animations first and foremost to be displayed on massive, stage displays. I was just wondering if anyone had any clever ideas on how to make this work without sacrificing too much quality.

10

Yeah, it looks like on of those irrational demands by people who don’t know how things work.

A long way ago I had a client who wanted to have a searchable database on a DVD that you’d put into a DVD player and search using your remote :rofl:

It’s our obligation to tell people like that that things simply won’t work that way and give them technical specifications to conform to.

11

Small update, artist said they’ll try to export each individual layer separately, so that might help me out.

12

The “right” way to do it would be artist adapting the whole thing for the game in close communication with you.

13

Then I sincerely doubt you will ever get it to work well. Not because you don’t have skills, but because you are working with assets that were designed for a completely different purpose.

14

So I’ve been messing about and came up with a nice middle ground.

I created a much smaller resolution video file (ogv) and scaled it up to be the same size as the main artwork. This video plays when the zoom level of the camera is below a certain point (so it’s zoomed out far enough).
There’s a second, hidden layer that I created, which consists of 3 different Sprite2Ds, and using magick, I spliced the main, high resolution artwork into 3 different parts, and on the scene itself, I put them together again.
As the player zooms in the camera, I fade out the video player and fade in the Sprites instead, making sure to play around with the colors a little to make it as seamless as possible.

This way I have the best of both worlds, we still have a nice, animated background, but if the player wants to see more details and zooms in, I pause the player and swap it out with the high resolution, but static Sprites, so they can see more details.

Leaving this open but also putting this here in case anyone in the future might need to deal with something like this as well.

PS: I wish I could show it in action because the art looks so cool, but I need to wait another 2 months before I can…

15

Make sure to test with different devices, and many mid-range. That frame size will make some OpenGL ES drives freakout, as it breaks 8192x8192 in one dimension.

Each of your frame is 135.6 megs in size btw (at 4 bytes per pixe, uncompressed). You’re going to do a LOT of swapping to run all those anim frames on Android handhelds, aren’t you ?

I’m curious about the performance on your average phone.

Cheers, break a leg and all that!

16

My current solution completely solves that. There are two streams at the moment, one of them I rendered with a maximum resolution of 2000 on either side, that will be automatically playing on low end devices, and the second video has a maximum resolution of 4000 pixels on either dimensions.

And I know about the texture limit, I ran into it a lot last year with a different game. That’s why I took the main, high resolution static image into 3 different sprites, that way, it looks like one massive artwork, and the resolution is huge, but no single texture is bigger than 4000 pixels at all.

PS: My current low end device that I test with is a Samsung Galaxy A20e and it works quite well on there, so I’m happy!

17

We faced a similar problem with Dicey Dungeons (custom engine) All the animations were authored in After Effects and there were a lot of animations (characters, enemies, backgrounds, effects,…).

We ended using bodymovin (now called Lottie) to export the after effects animations to json and a bunch of spritesheets and I wrote an importer and renderer for those. I had to manually rasterize some of the parts of the animations when some effects were applied to them or ask the artist if it was possible to change the animation to avoid using those effects if the rasterized size was too big.

I found this plugin for godot

But lacks mobile support as far as I can tell.

Here are the notes I left to the team in case they wanted to add/modify animations:

Notes

Preparation

  • Open after effects and check that the bodymovin extension has been installed correctly in Window->Extensions->Bodymovin and enable it.
  • Go to Edit->Preferences->General and enable Allow Scripts to Write Files and Access Network

Exporting the animation

  • Open the animation you want to export.
  • Look through all the animation layers and compositions and their layers for the different effects they use. Check the what to do with this effect section to know how to deal with it.
  • Once you have dealt with the effects check if the layers come from a “video” source or not in the Project tab. They are usually inside a folder and have the photoshop icon. If the type doesn’t say Photoshop they are considered “video” sources. In that case, open the photoshop file open the Timeline tab in Window->Timeline and it should have a Convert button. Press it and save the psd file. This will convert them back to “image” sources and will export them while exporting the animation.
  • Go back to after effects and check that the last step worked. If it still says that they are “video”, close and open it again.
  • In after effects, open the Bodymovin tab and:
    • Locate and select the main composition in the list.
    • Select it.
    • Open its settings (click in the cog icon next to it)
    • Open the Assets section.
    • Enable Original Asset Names
    • Click Save
    • Select the destination folder clicking on the ... next to it
    • Click Render
  • If everything has gone okay, you should have a data.json file and a images folder filled with the images used in the animation in your selected destination folder.

Packaging the files

  • Open TexturePacker
  • Set the Data Format to Sparrow / Starling
  • Add all the images that need to be packed (if some are part of a rasterized layer you don’t need to add them). Don’t use folders, just drag and drop the png files.
  • Under Layout set the Max size to 4096x4096
  • Open the Scaling variants popup.
  • Set the first one:
    • Variant name: 4k
    • Scale: 1
    • Max. texture size: 4096x4096
  • Add a second one:
    • Variant name: 1080
    • Scale: 0.5
    • Max. texture size: 2048x2048
  • Open the Data file File selector and navigate towards your destination folder. Save it as pack_{v}.xml
  • In the same input Data file remove the xml extension and use pck
  • Save your project to the same destination folder as before.
  • Publish the sprite sheet
  • Now you should have in your destination folder data.json, pack_4k.pck, pack_4k.png, pack_1080.pck and pack_1080.png
  • Copy those files into the game’s data folder

What to do with this effect

Any effect that needs to be rasterized or that’s applied to a composition layer and needs to be rasterized will be marked with (!!!) Check the section oh god why to know how to deal how to rasterize a layer

  • Bezier warp (!!!)

    • While this effect is implemented in the game, some configurations aren’t:
      • If the effect’s bounding box is the same size as the image it should work without any problem in the game.
      • If the effect’s bounding box is bigger than the image then you will need to rasterize it. (!!!)

  • Wiggle -

    • Open the (Transform) effect under the Wiggle one.
    • Select all the parameters that have extensions (in red)
    • Go to Animation->Keyframe Assistant->Convert Expression to Keyframes

  • Liquify (!!!)

  • Any other effect (!!!)

  • If the layer has the T (next to Mode) checkbox enabled, it whole composition will need to be rasterized (!!!)

oh god why (how to rasterize a layer)

I’m sorry you reached this path. Be prepared to lose some sanity in this process. Get a notepad or some text editor ready.

  • Before going through all the stuff get the script and render settings ready:
    • Select a layer.
    • Go to File->Scripts->Run Script File...
    • Locate and select the script rd_RenderLayers.jsx (it will be accessible later in the File->Scripts->Recent Scripts Files section)
    • Click on Queue only because we aren’t actually rendering it but getting everything ready.
    • It will open the Render Queue tab with a new job added
    • Click in the small icon to the right of Render Settings: and select Make Template...
    • Set it a name (I used 10fps)
    • Click on the Edit... button.
    • Select Use this frame rate: and set it to 10
    • Click Okay and Okay
    • Close after effects and open it again because I have no idea how to remove a job from the render queue
  • Rendering the layer:
    • Open the layer dropdown (Click on the arrow next to the layer)
    • Open its Transform
    • Remove all the transformation keyframes.
    • Select the layer.
    • Run the rd_RenderLayers.jsx script
    • Set the Render Settings Template to the template you created before.
    • Set the Output Module Template to PNG Sequence
    • Set the Output Folder to any folder you want (I use the same destination folder as the animation file but under sequences)
    • Set the Output Name Template to <prefix>_[####].png where prefix can be anything (I use seq_<name_of_layer> for that part)
    • Click Render
    • Write down the layer number, the prefix and the fps you used. You’ll need them later.
  • Adding the metadata to the animation:
    • Go to the first keyframe of the animation
    • Deselect any layer you had selected.
    • Press the * key. It will create a marker (it’s a small grey triangle at the top of the timeline). If the marker is created in a layer you didn’t really deselected all the layers. Delete it and try again.
    • Double-click on it and fill the Comment section with this JSON structure (remap is an array so add as many JSON objects as needed inside it):
    	{
		"remap": [
			{
				"layer":6, 				// The layer number in the composition
				"target":"seq_bg_",		// The prefix you used when rendering the layer
				"fps":10, 				// The fps
				"parent":"comp_0",		// If the layer was inside a composition that isn't the main one, use the id of the exported composition (explained below)
				"fix_position":false	// Leave it false unless it doesn't work correctly in the game then try true. If it doesn't work either let me know so we can cry together.
			}
		]
	}
    • If the rendered layer was inside a composition that isn’t the main one you will need to export the animation before with the bodymovin extension and look for the id inside the data.json file. It will be under the assets array and the id should start with comp_