Network

Shotgun Software

www.shotgunsoftware.com

Shotgun Software has agreed to let us use their service for free for the remainder of the production. Here is going to be some information regarding what is done on the server and to why it is done this way. Since we are starting to use the service midway through production, the setup is going to be a bit ad-hoc however it will be a much better way of knowing exactly what is being done, changed, redone etc…


Asset Managment

http://www.animationpipeline.com/

Over the course of the production of this short asset management has been a severe issue, going from the Drop service, to FTP, to Unison (with the custom Arachnid tool) to now finally using SVN.

Current

SVN : CURRENT Asset management


DropBox service : Was the primary tool (soon to be depreciated)
Arachnid : Unison based file sync system developed for this project.
FTP Server : A private server that serves as backup.
Animux SVN : Final Renderman assets (RIB, shaders, textures) upload


Our SVN server is being hosted by NOYX Studio and providing the administration for this server. This server has been a test bed for not only FTP but also with Arachnid and now finally SVN. We will continue to be using this server through out the rest of the production.

In addition to the SVN service, one more tool has been added to our collection and that is Blender-Aid. This tool is a python server that you can connect to via a web browser that shows you all the files associated with a particular project, primarily Blender files. Not only that but it can give you a detailed list of linked data blocks, can move and rename these files and fix broken links. There is a test currently to see if this would serve useful on our SVN server or if we can just keep that tool local to each users system. While it does not serve much more than a tool for fixing links, it is a nice tool to have when you have to deal with hundreds of files.


FTP Backup

FTP backup has been revamped as well, using a more automated system rather than manual as it has been so far. Either using a cron job or by entering a command, either method will be sent via SSH to the server itself, replacing changed files and adding new ones. There will be a backup version that will be added to a tar in case of emergency as well so we can rollback if needed.

Our Animux SVN will most likely still be used for the final RIB files when we go into final rendering since it would be far easier to export from Mosaic locally into a RIB render specific directory rather than trying to accomplish this remotely via VNC client. That process will have to be worked out at a later date.


RedWidow

RedWidow is a "classified" project that is going to replace the current Shotgun Software pipeline web application as an in-house tool in the future.


Arachnid (OBSOLETE)

Arachnid is a simple wrapper script around the Unison file syncing tool. It is operated from the command line, bash under Linux, Cygwin under Windows. It is remaining here for informational purposes now.

Preparation

You must have a working installation of Unison within a bash environment. Bash is usually available on most Linux distributions, and is often the default shell. Under Windows, it will be necessary to install the Cygwin environment, as, while Unison is available in native Windows, arachnid uses various bash/linux tools to operate. Installation of Cygwin is straightforward, simply download the setup application from here, install the default options, and add unison, it should install version 2.27.57.

Once you have a running Unison install, extract the zip file of arachnid (location to be confirmed) into a suitable folder on your machine, this folder will become the project folder for Project Widow, all files will be synced into here.

First Use

Once your system is prepared, simply run arachnid as follows.

./arachnid sync

This will bring the current state of the project files from the server to your local folder.

General Use

Arachnid is a simple script that takes a command and a number of options. Calling arachnid with the single command help will display the basic use and list of commands, calling with help command will show specific help for the specified command.

Commands

sync

This will synchronize any changes either local or on the server. Under normal operation, the command on it's own will simply list all changes along with information about where the change is, local or on the server, and at the end ask for confirmation of the sync. This is the most common situation, however, if a file has been changed locally and remotely, the listing will stop at that point and ask how to resolve the conflict. Unison is waiting for a keypress to tell it what to do, pressing '?' will list the options, common ones are '>' to force Unison to copy from local to the server, overwriting changes on the server, '<' to force the server changes to overwrite local changes, or '/' to skip. If you specify the option -i after the sync command, Unison will ask what to do for all changes, not just conflicts. After the sync command, if any files are listed, only those files will be synced. The files can be specified relative to the project folder, or relative to the current directory, arachnid will try to identify the intent and sync the correct files.

As an example, consider the following directory structure.

/home/user/project_widow
  |
  +--dir_a
  |     |
  |     +--file_a
  |
  +--dir_b
        |
        +--file_a

From within the folder /home/user/project_widow/dir_a, you can run arachnid sync dir_a/file_a or arachnid sync file_a to achieve the same results.

The output of Unison will look as follows.

local          server
changed ---->            dir_a/file_1
        <----  changed   dir_a/file_2
changed <-?->  changed   dir_a/file_2

Where the first example shows a local change, the second a server change and the third a conflict (both local and server change).

update

This command will update the arachnid system itself, files which are normally ignored during a project sync command. This will only rarely be required, usually only when one of the admins makes a change to arachnid, and it will usually be communicated via mailing lists or forums that an update is necessary. Note: as the update command works on inherent knowledge of the files that constitute arachnid, it may occasionally be necessary to call arachnid update twice in succession, as the first call will only update files that it currently knows about, the second will then include new files. Again, if a dual update is required, this will be communicated by the admins.

reset

Arachnid keeps some information in cache files locally as part of it's normal operation. Occasionally it will be necessary to regenerate these caches, specifically if the server addresses change, this command forces the cache to be regenerated. Once again, this is a rarely needed command, and will only be needed when instructed by an admin.


Open Movie?

Also an idea of following the open movie method, we might end up making all the main assets available for download from this very FTP server. Since the server is meant for an audio studio, the server has a system that places updated or new files in particular directories that show up on the website for public view. All that needs to be done is a new FTP page and anything that is in the backup server will be available for future public download. This will only come online once the production has been completed however.


LOGS

We are trying to keep logs of changes. Of course this always can be overlooked by all so please try to remember to record changes in the MAIN CHANGE LOG.

One neat feature of Blender is that the text editor can also serve as a log file for that file in particular. This makes it very easy for people to make changes and then make note of it on that file, as opposed to adding changes to a general log file in the project folder. Plus in the internal log it is noted for that specific file.


Bug Reporting

If encountering bugs related to Mosaic or Aqsis please make sure to zip (or tar) up the Mosaic export directory and place them into /ProjectWidow/RIB_BUGS folder. If your project folder is filled with extra unrelated files, go into Mosaic and press 'Purge Export Project Folder' and then re-render the bug in question.

This way both Eric, Paul and Chris can get the exported data and examine the problem. If they need additional info or files they will let us know.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License