R – How to use subversion to deploy web applications

perlsvnweb-applications

We are a small team of 4 developers working on a web application. We use trac+svn on a shared server for version control and ticketing and we are happy and satisfied with this. The same shared server also hosts our web application. The application itself is a Perl CGI application that uses CGI::Application and a moderate number of standard (CPAN) and custom Perl modules that are installed in the usual (/usr/lib/perl…) and a few unusual locations (/home/user/lib/perl..). While the broad details might be irrelevant, the most important point is that the location/layout of libraries on our development machines is different from that on the production (shared) server. We have to live with this as a given. The library layout is identical on all development machines though.

Here is a typical, but clearly sub-optimal work-cycle that my colleagues and I follow:

Code and test on development machines
Checkout/Commit/Update our code onto the SVN
Periodically "svn export" onto the appropriate DocumentRoot of the server
Hand edit the exported tree to set the library includes match the library layout on the server
Test application on live server, raise tickets for each other
Go to 1

Clearly there must be a better way and would appreciate hearing from others who might be handling this better than we are. For example is there a way to svn export and fix the library locations in an automated way? Or is there some completely different way to handle this situation than we have been doing so far.

Thank you for your attention

Best Solution

You should have scripts that do this for you that can be run from a local box. Mine always look something like:

$> checkout from source or copy from working
$> run sed/perl -pi/copy to convert configs to the production values 
     (ie cp production.config myconfig)
$> upload to web server (rsync/ssh/ftp/etc)
$> ssh $SERVER migrate_db, set permissions, run unit tests, etc

The last one requires ssh access which I always look for but everything else can be done locally. You'd usually have a set of dev configs and a set of production configs (or a script to convert from dev to production

One step uploads are always a really good idea.

Marking files as ignored:

By "ignored file" I mean the file won't appear in lists even as "unversioned": your SVN client will pretend the file doesn't exist at all in the filesystem.

Ignored files are specified by a "file pattern". The syntax and format of file patterns is explained in SVN's online documentation: http://svnbook.red-bean.com/nightly/en/svn.advanced.props.special.ignore.html "File Patterns in Subversion".

Subversion, as of version 1.8 (June 2013) and later, supports 3 different ways of specifying file patterns. Here's a summary with examples:

1 - Runtime Configuration Area - `global-ignores` option:

This is a client-side only setting, so your global-ignores list won't be shared by other users, and it applies to all repos you checkout onto your computer.
This setting is defined in your Runtime Configuration Area file:
- Windows (file-based) - C:\Users\{you}\AppData\Roaming\Subversion\config
- Windows (registry-based) - Software\Tigris.org\Subversion\Config\Miscellany\global-ignores in both HKLM and HKCU.
- Linux/Unix - ~/.subversion/config

2 - The `svn:ignore` property, which is set on directories (not files):

This is stored within the repo, so other users will have the same ignore files. Similar to how .gitignore works.
svn:ignore is applied to directories and is non-recursive or inherited. Any file or immediate subdirectory of the parent directory that matches the File Pattern will be excluded.

While SVN 1.8 adds the concept of "inherited properties", the svn:ignore property itself is ignored in non-immediate descendant directories:

cd ~/myRepoRoot                             # Open an existing repo.
echo "foo" > "ignoreThis.txt"                # Create a file called "ignoreThis.txt".

svn status                                  # Check to see if the file is ignored or not.
> ?    ./ignoreThis.txt
> 1 unversioned file                        # ...it is NOT currently ignored.

svn propset svn:ignore "ignoreThis.txt" .   # Apply the svn:ignore property to the "myRepoRoot" directory.
svn status
> 0 unversioned files                       # ...but now the file is ignored!

cd subdirectory                             # now open a subdirectory.
echo "foo" > "ignoreThis.txt"                # create another file named "ignoreThis.txt".

svn status
> ?    ./subdirectory/ignoreThis.txt        # ...and is is NOT ignored!
> 1 unversioned file

(So the file ./subdirectory/ignoreThis is not ignored, even though "ignoreThis.txt" is applied on the . repo root).

Therefore, to apply an ignore list recursively you must use svn propset svn:ignore <filePattern> . --recursive.
- This will create a copy of the property on every subdirectory.
- If the <filePattern> value is different in a child directory then the child's value completely overrides the parents, so there is no "additive" effect.
- So if you change the <filePattern> on the root ., then you must change it with --recursive to overwrite it on the child and descendant directories.
I note that the command-line syntax is counter-intuitive.
- I started-off assuming that you would ignore a file in SVN by typing something like svn ignore pathToFileToIgnore.txt however this is not how SVN's ignore feature works.

3- The `svn:global-ignores` property. Requires SVN 1.8 (June 2013):

This is similar to svn:ignore, except it makes use of SVN 1.8's "inherited properties" feature.
Compare to svn:ignore, the file pattern is automatically applied in every descendant directory (not just immediate children).
- This means that is unnecessary to set svn:global-ignores with the --recursive flag, as inherited ignore file patterns are automatically applied as they're inherited.

Running the same set of commands as in the previous example, but using svn:global-ignores instead:

cd ~/myRepoRoot                                    # Open an existing repo
echo "foo" > "ignoreThis.txt"                       # Create a file called "ignoreThis.txt"
svn status                                         # Check to see if the file is ignored or not
> ?    ./ignoreThis.txt
> 1 unversioned file                               # ...it is NOT currently ignored

svn propset svn:global-ignores "ignoreThis.txt" .
svn status
> 0 unversioned files                              # ...but now the file is ignored!

cd subdirectory                                    # now open a subdirectory
echo "foo" > "ignoreThis.txt"                       # create another file named "ignoreThis.txt"
svn status
> 0 unversioned files                              # the file is ignored here too!

For TortoiseSVN users:

This whole arrangement was confusing for me, because TortoiseSVN's terminology (as used in their Windows Explorer menu system) was initially misleading to me - I was unsure what the significance of the Ignore menu's "Add recursively", "Add *" and "Add " options. I hope this post explains how the Ignore feature ties-in to the SVN Properties feature. That said, I suggest using the command-line to set ignored files so you get a feel for how it works instead of using the GUI, and only using the GUI to manipulate properties after you're comfortable with the command-line.

Listing files that are ignored:

The command svn status will hide ignored files (that is, files that match an RGA global-ignores pattern, or match an immediate parent directory's svn:ignore pattern or match any ancesor directory's svn:global-ignores pattern.

Use the --no-ignore option to see those files listed. Ignored files have a status of I, then pipe the output to grep to only show lines starting with "I".

The command is:

svn status --no-ignore | grep "^I"

For example:

svn status
> ? foo                             # An unversioned file
> M modifiedFile.txt                # A versioned file that has been modified

svn status --no-ignore
> ? foo                             # An unversioned file
> I ignoreThis.txt                  # A file matching an svn:ignore pattern
> M modifiedFile.txt                # A versioned file that has been modified

svn status --no-ignore | grep "^I"
> I ignoreThis.txt                  # A file matching an svn:ignore pattern

ta-da!

How to edit log message already committed in Subversion

Essentially you have to have admin rights (directly or indirectly) to the repository to do this. You can either configure the repository to allow all users to do this, or you can modify the log message directly on the server.

See this part of the Subversion FAQ (emphasis mine):

Log messages are kept in the repository as properties attached to each revision. By default, the log message property (svn:log) cannot be edited once it is committed. That is because changes to revision properties (of which svn:log is one) cause the property's previous value to be permanently discarded, and Subversion tries to prevent you from doing this accidentally. However, there are a couple of ways to get Subversion to change a revision property.

The first way is for the repository administrator to enable revision property modifications. This is done by creating a hook called "pre-revprop-change" (see this section in the Subversion book for more details about how to do this). The "pre-revprop-change" hook has access to the old log message before it is changed, so it can preserve it in some way (for example, by sending an email). Once revision property modifications are enabled, you can change a revision's log message by passing the --revprop switch to svn propedit or svn propset, like either one of these:
$svn propedit -r N --revprop svn:log URL 
$svn propset -r N --revprop svn:log "new log message" URL 
where N is the revision number whose log message you wish to change, and URL is the location of the repository. If you run this command from within a working copy, you can leave off the URL.

The second way of changing a log message is to use svnadmin setlog. This must be done by referring to the repository's location on the filesystem. You cannot modify a remote repository using this command.
$ svnadmin setlog REPOS_PATH -r N FILE
where REPOS_PATH is the repository location, N is the revision number whose log message you wish to change, and FILE is a file containing the new log message. If the "pre-revprop-change" hook is not in place (or you want to bypass the hook script for some reason), you can also use the --bypass-hooks option. However, if you decide to use this option, be very careful. You may be bypassing such things as email notifications of the change, or backup systems that keep track of revision properties.

Best Solution

Related Solutions

How to ignore files in Subversion

Marking files as ignored:

1 - Runtime Configuration Area - global-ignores option:

2 - The svn:ignore property, which is set on directories (not files):

3- The svn:global-ignores property. Requires SVN 1.8 (June 2013):