Capture your Mobile Application's Network Logs from Automation - Part 1

Capture your Mobile Application's Network Logs from Automation - Part 1

Intercept apps network logs using MITM Proxy

Himanshu Nikhare's photo
Himanshu Nikhare
·

3 min read

Table of contents

One of the biggest challenges for automation engineers is to identify the root cause of their test failures, and if we are talking about mobile automation, it's almost a nightmare to debug issues when some background service fails.

The articles help us solve this issue by allowing us to capture real-time network logs from Appium Automation Test Framework which executes cases parallelly on the selenium grid.

What are we going to cover?

  • Connect MITM Proxy with Android/IOS Device (Part 1)

  • Save real-time network logs to dump file (Part 2)

  • Add automation run specific network logs to our HTML report (Part 3)

What is MITM Proxy?

Man in the Middle Proxy is an interactive proxy that provides a set of tools that can be used to intercept, inspect, modify, and replay network web traffic such as HTTP/1, HTTP/2, WebSockets, or any other SSL/TLS-protected protocols.

Why are we using this Proxy?

The proxy comes with tons of powerful features which allow us to intercept and modify requests in almost any way possible.

The proxy consists of 3 powerful front-end tools

  • mitmproxy is an interactive, SSL/TLS-capable intercepting proxy with a console interface for HTTP/1, HTTP/2, and WebSockets?

  • mitmweb is a web-based interface for mitmproxy.

  • mitmdump is the command-line version of mitmproxy.

6 Different modes of operation

Easy installation of Certificates using: mitm.it magic domain, numerous ways to create custom addons, many event hooks and so much more, you can read more about these features on their official documentation at: https://docs.mitmproxy.org/stable/

Connect MITM Proxy with Android/IOS Device

Install MITM Proxy

Use brew if using Mac

brew install mitmproxy

For download instructions for other Operating Systems please follow: https://docs.mitmproxy.org/stable/overview-installation/#installation

Starting Proxy Server

We will be using mitmdump to start our proxy server

mitmdump --listen-port 8090 "domain-name.com"

Configure proxy settings on your mobile device

Find your private IP

Mac/Linux

ifconfig | grep -i mask

From the list of output, IP Starting with 192.168.XXX.XXX is your private IP

Windows

Search for cmd in the Windows search bar, then in the command line prompt, type ipconfig to view the private IP address.

Are you using a real device?

No, using an emulator

Open your emulator using below command or add proxy settings from emulator settings

emulator -avd <EMULATOR_NAME> -http-proxy http://0.0.0.0:8090

Connect your device to the same network as the proxy device

  • Open Network Settings -> Goto Wifi

  • Open Network Details

  • Edit connection and configure proxy settings, make sure to put the correct private IP Address and port

  • Once connected verify by opening mitm.it in your browser, if everything is working fine you will see a page like in the below image

You will be able to intercept HTTP Requests till this part, if you are trying to intercept HTTPS Requests you need to install Mitm CA Certificate in your system. Make sure your app respects the user-installed CAs.

Your trusted Certificate Authorities (CAs) are the organizations that you trust to guarantee the signatures of your encrypted traffic and content.

  1. Make application respect installed ca certificate (not preferred for production build)

in network_security_config.xml

<network-security-config xmlns:tools="http://schemas.android.com/tools"
    tools:ignore="MissingDefaultResource">
    <debug-overrides>
        <trust-anchors>
            <certificates src="system" />
            <!-- Trust user added CAs while debuggable only -->
            <certificates src="user" />
        </trust-anchors>
    </debug-overrides>
</network-security-config>

  1. Install MITM CA Certificate on your device

    You can download CA by visiting mitm.it, you cannot install CAs directly and hence needed to be installed from Settings

    Settings -> Security -> Advanced -> Encryption and credentials -> Install a certificate -> CA certificate -> VPN and apps -> OK

You are ready to intercept HTTPS Requests now, Open you app and enjoy intercepting

But this does not finishes our job, part-2 continues with instructions on how we can connect our framework with MITM Proxy.

appiumAndroidiOSTestingautomation