502 Bad Gateway, in Response to SVN COPY with SSL off-loading

Diagram of a reverse proxy.

Diagram of a reverse proxy.

This post helps in fixing 502 Bad Gateway error caused by reverse proxy, during an SVN COPY operation.

Command: Commit
Adding: C:\Project\Scripts\00_create_objects.xml
Error: Commit failed (details follow):
Error: Server sent unexpected return value (502 Bad Gateway) in response to COPY
Error: request for ……

If renaming and coping files on Subversion are resulting in this error, then it implies you are accessing Subversion over HTTP/HTTPS with WebDAV, which is an extension to HTTP protocol.

The Basics of SVN Copy

So basically it means that accessing Subversion over HTTP/HTTPS is just like browsing sites where you send HTTP-GET and HTTP-POST requests to a web server. Subversion can be accessed over HTTP/HTTPS by integrating it with Apache Web server. Apache web server facilitates the HTTP interface for Subversion. However, along with these standard HTTP-Methods like GET, POST, DELETE, PUT, OPTIONS, there are more HTTP-Methods used to access Subversion over HTTP. These HTTP-Methods like COPY, PROPFIND, PROPPATCH, LOCK, MOVE etc are part of WebDAV extension for HTTP.

WebDAVCOPY, or simply HTTP-COPY method creates a duplicate of a resource on server. Before renaming a file SVN client sends a request to make a copy of that file to new name. So this method is used.

Source of the copy operation is the URL itself.

Destination of the copy operation is a HTTP-Header “Destination: “.

So a HTTP-COPY request looks like this:

COPY /REPO/PROJECTS/!svn/bc/22751/WORK/test.sql HTTP/1.1
User-Agent: SVN/1.6.13 (r1002816)/TortoiseSVN-1.6.11.20210 neon/0.29.4
Connection: TE
TE: trailers
Host: labs.mydomain.org
Accept-Encoding: gzip
Destination: https://labs.mydomain.org/REPO/PROJECTS/!svn/wrk/854kjkj-97bc-3745-84dc-a9a8dee4d381/WORK/test.exclude
Depth: 0
Overwrite: T
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops
Content-Length: 0
Accept-Encoding: gzip
Authorization: Basic HJKNSKfjU9433c2JAMTIz
X-Forwarded-For: 192.168.1.34
Connection: close

Here test.sql is being renamed to test.exclude.

The Problem

When accessing SVN over HTTPS, HTTPS can be terminated on Apache web server or it can be off-loaded to different software like Reverse Proxy. In this case HTTPS requests are first received by reverse proxy and then a new HTTP request is forwarded to web server. This HTTPS-HTTP conversion saves resources on web server required for SSL processing. However reverse proxies only rewrite URIs by default. They don’t change the headers, but they can. Problem is that HTTP-COPY request’s URI is changed to HTTP, but the “Destination:” header’s URI starts with HTTPS. When web server finds URI scheme of source and destination different, it throws an error 502 (Bad Gateway). This basically means that the HTTP-COPY destination is located on a different server and thus refuses to accept the resource.

The Solution

So now we know the problem. There are two basic approaches to solve the problem. One is we rewrite “Destination:” header on Apache web server and the second is that we rewrite “Destination:” header on reverse proxy.

#1 : Rewrite “Destination:” Header on Apache

Step # 1: Enable Module “headers_module”

Add following line or uncomment if commented.

LoadModule headers_module modules/mod_headers.so

Step # 2: Rewrite

Add following line outside Location Directive.

RequestHeader edit Destination ^https: http: early

Step # 3: Restart Apache

#2 : Rewrite “Destination:” Header in Reverse Proxy

It may not be always possible to rewrite “Destination” header in Apache. For example above solution will not work with Apache 2.0. “Edit” action for “RequestHeader” directive is supported in version 2.2 and above.

In my case reverse proxy being used was HAProxy. HAProxy is an excellent solution offering load balancing, high-availability, reverse proxy and even SSL off-loading. It can terminate SSL and forward non-SSL requests to servers. We can use HTTP Header Manipulation feature of HAProxy to rewrite “Destination:” header.

Step # 1: Edit HTTP header “Destination:” in HAProxy

Add following line to the frontend section of HAProxy where SSL is processed.

reqirep Destination:\ https(.*) Destination:\ http\\1

Step # 2: Check for errors in HAProxy’s configuration

haproxy -f haproxy.conf –c

Step # 3: Restart HAProxy

rc.d/haproxy restart
About Dinesh Sharma

Experienced system architect, programmer, and trainer. This blog is a way of giving back and helping the community. So feel free to ask a question or to leave a comment.

  • Ted FIsher

    Thank you for this Dinesh. It provided a quick and helpful way to resolve our experience with the exact same problem behind a Cisco ACE load balancer.
    [email protected]

    • I am glad to know that Ted. Thanks for leaving the comment. I really appreciate it.

  • rajneesh

    Thanks, the info was very helpful. We use AWS ELB in front to terminate SSL. This re-configuration was required in the Apache server behind ELB.

    • Thanks Rajneesh for leaving a comment. I appreciate.

  • Pingback: SVN, COPY and ’502 Bad Gateway’ | Pierre Roudier's Blog()

  • andy

    RequestHeader apache directive should be added outside Location block (e.g. on VirtualHost level) otherwise will not work, see apache docs.

    • Thanks Andy,
      Yes early directive should be outside location block. Updated 🙂

  • mmm

    Had the same problem. This page was result #1 on google. I spent two minutes to understand the problem from your nice description. Then two more to fix it. Worked like a charm. Thanks!

  • Martin

    Thanks, helped me solve the same issue I had behind nginx, http://mailman.nginx.org/pipermail/nginx/2007-January/000504.html gave me the code to do the rewrite on the nginx server (although the apache changes you suggested worked too)

  • Pingback: SVN & IIS HTTPS offloading reverse proxy: Fix “Server sent unexpected return value (502 Bad Gateway) in response to COPY request” | Jan Jonas' blog()

  • Sanjay Patil

    Thank you for this blog! 🙂

  • Brady Vidovic

    Thank you! Fixed it exactly (using HAProxy as well)

  • Ciaran

    Thanks Dinesh, fixed my problem with Apache/SVN behind an NGINX proxy

  • Samuel Lopes Grigolato

    Thanks! That solved my problem! I’d just like to point out that this, with little modification, solves the other way around too: accessing the proxy via HTTP which in turn access the SVN server via HTTPS.

  • Igor Astakhov

    for nginx:
    set $dest $http_destination;
    if ($http_destination ~ “^https://(.+)”) {
    set $dest http://$1;
    }

    proxy_set_header Destination $dest;

  • Vivek

    Your solution is awesome… Thank you the first one worked for me

  • Pingback: 502 Bad Gateway, in Response to SVN COPY with SSL off-loading | 이슬나라()

  • Angelo Quaglia

    Very useful, many thanks indeed!

  • Sourabh Chowdhary

    Thanks Dinesh for this useful information. I have also found this link:

    http://www.justinsilver.com/technology/svn-copy-502-bad-gateway-error

  • Shrirang Patel

    Hi Dinesh

    We use KEMP Load Balancer. What setting do i need to make on this load balancer?

  • Eric Covener

    Worked for me, great writeup

  • David Česal

    Thank you! Your solution (#1) worked like a charm!

  • Mike Shaw

    Thank-you Dinesh! This information made it easy to fix this issue when when moved our SVN server behind a reverse proxy server.