How to Setup Visual Studio (TFS) Test Agents in the Cloud

We ran into some issues trying to get the Visual Studio Test Agents to register and communicate with the Visual Studio Test Controller when the controller is inside our network, and the agents are in the cloud. There are some strange quirks that need to be addressed before this will work. However, following these steps you should be able to run test agents on VMs in Amazon AWS EC2, Microsoft Windows Azure, etc.

Steps:

1. You will need to NAT a public IP address to the private IP address of your Visual Studio Test Controller system, or place it in a DMZ and assign it a public IP address. I recommend the first option.
2. You will need to put in firewall rules to allow the following ports from the public IP address of your Visual Studio Test Agent VMs to your Visual Studio Test Controller system.
• 137/udp, 138/udp, 139/tcp, 445/tcp, 6901/TCP, ICMP echo request.
3. Create a domain user named "VsTestService" in the domain where the test controller resides.
• Add this user ( <domain>\VsTestService ) to the "administrators" group on your Visual Studio Test Controller system.
• Add this user ( <domain>\VsTestService ) to the "TeamTestAgentService" group on the Visual Studio Test Controller system.
4. Create a local account on the Visual Studio Test Controller system named "VsTestService" with the exact same password as the domain "VsTestService" account that you created in step 3.
• Add this user ( <ComputerName>\VsTestService ) to the "admininstrators" group on the Visual Studio Test Controller system.
• Add this user ( <ComputerName>\VsTestService ) to the "TeamTestAgentService" group on the Visual Studio Test Controller system.
• Add this user ( <ComputerName>\VsTestService ) to the "TeamTestControllerAdmins" group on the Visual Studio Test Controller system.
5. Add the netbios hostname of the Visual Studio Test Agent system to the hosts file on the Visual Studio Test Controller system with the public IP address Visual Studio Test Agent system (or add it to your internal DNS). You can get this by running "echo %computername%" at a command prompt.
• Keep in mind that the public IP address of these VMs in the cloud often change if you shut off the VM and later power it back on. In this case, you will need to update the hosts file (or internal DNS) each time the IP address changes.
6. Install the Visual Studio Test Controller on the Visual Studio Test Controller system.
• When configuring it, set the service to run under the domain service account you created ( <domain>\VsTestService ). If you run it under the local account, the test manager will likely fail to authenticate to the test controller.
7. You will need to put in firewall rules to allow the following port 6910/tcp from the public IP address of your Visual Studio Test Controller system to your Visual Studio Test Agent system.
8. Create a local account on the Visual Studio Test Agent system named "VsTestService" with the exact same password that you used when you created the "VsTestService" accounts in steps 3 and 4.
• Add this user ( <ComputerName>\VsTestService ) to the "admininstrators" group on the Visual Studio Test Agent system.
9. Add the netbios hostname and FQDN (active directory domain, not internet domain) of the Visual Studio Test Controller system to the hosts file of the Visual Studio Test Agent system with the public IP address of the Visual Studio Test Controller system (see the example below).

   #Example

   111.222.333.444     ControllerName

   111.222.333.444     ControllerName.ADDomain.local

    
10. Log off of the Visual Studio Test Agent system and log back on with the local "VsTestService" account you created ( <ComputerName>\VsTestService ).
11. Install the Visual Studio Test Agent on the Visual Studio Test Agent system.
• When configuring it, set the service to run under the local "VsTestService" account you created ( <ComputerName>\VsTestService ).

It should install and test successfully if you followed all of these steps correctly!

Note* If you have any hosts file entries with the IP address 127.0.0.1, it will probably still fail. For more information on this quirk, see this article.