502 Bad Gateway, in Response to SVN COPY with SSL off-loading

Diagram of a reverse proxy.

Diagram of a reverse proxy.

This post helps in fixing 502 Bad Gateway error caused by reverse proxy, during an SVN COPY operation.

Command: Commit
Adding: C:\Project\Scripts\00_create_objects.xml
Error: Commit failed (details follow):
Error: Server sent unexpected return value (502 Bad Gateway) in response to COPY
Error: request for ……

If renaming and coping files on Subversion are resulting in this error, then it implies you are accessing Subversion over HTTP/HTTPS with WebDAV, which is an extension to HTTP protocol.

The Basics of SVN Copy

So basically it means that accessing Subversion over HTTP/HTTPS is just like browsing sites where you send HTTP-GET and HTTP-POST requests to a web server. Subversion can be accessed over HTTP/HTTPS by integrating it with Apache Web server. Apache web server facilitates the HTTP interface for Subversion. However, along with these standard HTTP-Methods like GET, POST, DELETE, PUT, OPTIONS, there are more HTTP-Methods used to access Subversion over HTTP. These HTTP-Methods like COPY, PROPFIND, PROPPATCH, LOCK, MOVE etc are part of WebDAV extension for HTTP.

WebDAVCOPY, or simply HTTP-COPY method creates a duplicate of a resource on server. Before renaming a file SVN client sends a request to make a copy of that file to new name. So this method is used.

Source of the copy operation is the URL itself.

Destination of the copy operation is a HTTP-Header “Destination: “.

So a HTTP-COPY request looks like this:

COPY /REPO/PROJECTS/!svn/bc/22751/WORK/test.sql HTTP/1.1
User-Agent: SVN/1.6.13 (r1002816)/TortoiseSVN- neon/0.29.4
Connection: TE
TE: trailers
Host: labs.mydomain.org
Accept-Encoding: gzip
Destination: https://labs.mydomain.org/REPO/PROJECTS/!svn/wrk/854kjkj-97bc-3745-84dc-a9a8dee4d381/WORK/test.exclude
Depth: 0
Overwrite: T
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops
Content-Length: 0
Accept-Encoding: gzip
Authorization: Basic HJKNSKfjU9433c2JAMTIz
Connection: close

Here test.sql is being renamed to test.exclude.

The Problem

When accessing SVN over HTTPS, HTTPS can be terminated on Apache web server or it can be off-loaded to different software like Reverse Proxy. In this case HTTPS requests are first received by reverse proxy and then a new HTTP request is forwarded to web server. This HTTPS-HTTP conversion saves resources on web server required for SSL processing. However reverse proxies only rewrite URIs by default. They don’t change the headers, but they can. Problem is that HTTP-COPY request’s URI is changed to HTTP, but the “Destination:” header’s URI starts with HTTPS. When web server finds URI scheme of source and destination different, it throws an error 502 (Bad Gateway). This basically means that the HTTP-COPY destination is located on a different server and thus refuses to accept the resource.

The Solution

So now we know the problem. There are two basic approaches to solve the problem. One is we rewrite “Destination:” header on Apache web server and the second is that we rewrite “Destination:” header on reverse proxy.

#1 : Rewrite “Destination:” Header on Apache

Step # 1: Enable Module “headers_module”

Add following line or uncomment if commented.

LoadModule headers_module modules/mod_headers.so

Step # 2: Rewrite

Add following line outside Location Directive.

RequestHeader edit Destination ^https: http: early

Step # 3: Restart Apache

#2 : Rewrite “Destination:” Header in Reverse Proxy

It may not be always possible to rewrite “Destination” header in Apache. For example above solution will not work with Apache 2.0. “Edit” action for “RequestHeader” directive is supported in version 2.2 and above.

In my case reverse proxy being used was HAProxy. HAProxy is an excellent solution offering load balancing, high-availability, reverse proxy and even SSL off-loading. It can terminate SSL and forward non-SSL requests to servers. We can use HTTP Header Manipulation feature of HAProxy to rewrite “Destination:” header.

Step # 1: Edit HTTP header “Destination:” in HAProxy

Add following line to the frontend section of HAProxy where SSL is processed.

reqirep Destination:\ https(.*) Destination:\ http\\1

Step # 2: Check for errors in HAProxy’s configuration

haproxy -f haproxy.conf –c

Step # 3: Restart HAProxy

rc.d/haproxy restart
About Dinesh Sharma

Experienced system architect, programmer, and trainer. This blog is a way of giving back and helping the community. So feel free to ask a question or to leave a comment.