C# – Automating Mouse Movement and Clicks in Firefox programmatically

automationc++

I am trying to create an automated clicker program in C# to make some administrative tasks quicker. In short, I will be using one program (Firefox) and will need a program to automatically move the mouse and click wherever I tell it to.

At the moment I have this code, although I think I'm pretty far away from an actual solution.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.Drawing;

namespace Clicker
{
    public partial class Form1 : Form
    {

        [DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
        public static extern void mouse_event(long dwFlags, long dx, long dy, long cButtons, long dwExtraInfo);

        private const int MOUSEEVENTF_LEFTDOWN = 0x02;
        private const int MOUSEEVENTF_LEFTUP = 0x04;
        private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
        private const int MOUSEEVENTF_RIGHTUP = 0x10;

        public void DoMouseClick()
        {
            //Call the imported function with the cursor's current position
            Cursor.Position = new Point((int)10, (int)10);

            mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, Cursor.Position.X, Cursor.Position.Y, 0, 0);
        }

        public Form1()
        {
            //InitializeComponent();
        }
    }
}

I want my program to run on top of Firefox and once it has loaded to immediately start clicking in the positions I tell it to. It'll need to click thirteen 30×30 pixel buttons and will then stop, leaving me to finish the task.

How would I go about such a task?

EDIT: I forgot to mention that the front-end is in Flash. Weird, I know. Either way, it means that programs like Selenium won't work as its tracks page movements and clicks on links.

Best Solution

While this doesn't answer your question with C# code I would really have to recommend you look at AutoIt. It is a very simple scripting language for creating defined tasks to be repeated. I've used it successfully for a wide range of tasks from repeatedly entering in form data for testing web applications to fighting monsters in a MMO game.

Related Solutions

C# – Unity and ASP.NET WebForms – No parameterless constructor defined for this object

I see it has already been answered but just thought I would point out that you are synchronising all the calls to GetContainer with your locking pattern. A call to Application.Lock() actually takes out a write lock on the applicationState which is a singleton object in your web application and you will see issues if you want to scale this.

To tidy this up you could do a double checked lock. like this:

    public static IUnityContainer GetContainer(this HttpApplicationState application)
    {
        IUnityContainer container = application[GlobalContainerKey] as IUnityContainer;
        if (container == null)
        {
            application.Lock();
            try
            {
                container = application[GlobalContainerKey] as IUnityContainer;
                if (container == null)
                {
                    container = new UnityContainer();
                    application[GlobalContainerKey] = container;
                }
            }
            finally
            {
                application.UnLock();
            }
        }
        return container;
    }

I would also like to point out a neat pattern that we have used to ensure Controls and Pages have their Dependencies built up. We basically have a Generic PageBase and Generic ControlBase that all our pages and controls inherit from. I will just go into the pagebase as an example:

public abstract class SitePageBase<T> : SitePageBase where T : SitePageBase<T>
{
    protected override void OnInit( EventArgs e )
    {
        BuildUpDerived();
        base.OnInit( e );
    }

    protected void BuildUpDerived()
    {
        ContainerProvider.Container.BuildUp( this as T );
    }
}

Then in our Pages we can simply derive from Generic base and it will look after the build up.

public partial class Default : SitePageBase<Default>
{
        [Dependency]
        public IContentService ContentService { get; set; }

        protected override void OnPreRender( EventArgs e )
        {
            this.label.Text = ContentService.GetContent("labelText");
        }
     }

C# – Automating mouse clicks on screen

As you only wanted a general way, I didn't really make it perfect, but here is the idea:

Have a method for taking a screen shot:

public Bitmap ScreenShot()
{
    var screenShot = new Bitmap(Screen.PrimaryScreen.Bounds.Width,
                                Screen.PrimaryScreen.Bounds.Height,
                                PixelFormat.Format32bppArgb);

    using (var g = Graphics.FromImage(screenShot))
    {
        g.CopyFromScreen(0, 0, 0, 0, Screen.PrimaryScreen.Bounds.Size);
    }

    return screenShot;
}

And a method to find a certain color in a bitmap: Note that this implementation can be DRASTICALLY improved using unsafe code and LockBits (read here and here).

public Point? GetFirstPixel(Bitmap bitmap, Color color)
{
    for (var y = 0; y < bitmap.Height; y++)
    {
        for (var x = 0; x < bitmap.Width; x++)
        {
            if (bitmap.GetPixel(x, y).Equals(color))
            {
                return new Point(x, y);
            }
        }
    }

    return null;
}

Another method you'll need is one for clicking a certain point:

[DllImport("user32.dll",
           CharSet=CharSet.Auto,
           CallingConvention=CallingConvention.StdCall)]
private static extern void mouse_event(long dwFlags,
                                      long dx,
                                      long dy,
                                      long cButtons,
                                      long dwExtraInfo);

private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;

public void Click(Point pt)
{
    Cursor.Position = pt;
    mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, pt.X, pt.Y, 0, 0);
}

And finally, one to wrap it all up:

public bool ClickOnFirstPixel(Color color)
{
    var pt = GetFirstPixel(ScreenShot(), color);

    if (pt.HasValue)
    {
        Click(pt.Value);
    }

    // return whether found pixel and clicked it
    return pt.HasValue;
}

Then, the usage would be:

if (ClickOnFirstPixel(Color.Red))
{
    Console.WriteLine("Found a red pixel and clicked it!");
}
else
{
    Console.WriteLine("Didn't find a red pixel, didn't click.");
}

Best Solution

Related Solutions

C# – Unity and ASP.NET WebForms – No parameterless constructor defined for this object

C# – Automating mouse clicks on screen

Related Question