Desktop Automation
Introduction
Automation is a part of babblevoice Desktop. Automation is about registering a hotkey, so that an agent can quickly perform a task. That is, you can configure a keyboard shortcut such as ctrl-1 to dial the contact your agent has on their screen. We aim to provide functions so that you can integrate with your business function to automate tasks. We provide a sample/default script which can dial from EMIS Web/SystmOne and other text controls.
Please note, this interface is provided as is and without warranty. It is an interface we will continue to develop, but this is in beta, and you are responsible for any code you write against this interface.
Beneath the hood, we have provided a scripting interface to allow users to script automation tasks. This allows admins to design scripts to allow environments, such as call centres, to integrate telephone calls into workflow driving data in and out of the business application.
Scripts are written in Lua (www.lua.org). There is plenty of documentation available to learn this language if you do not know it already. As part of our interface, we aim to provide the interfaces into the Windows operating system to hook into Windows applications and then also into babblevoice to make calls, and when receiving calls do something with that information.
The babblevoice config file contains the reference to the automation script file, and in the program files folder there is a default script you can build your own scripts on.
Example, register a system hotkey (so that a user can hit ctrl+shift-b in any application), then toggle whether the babblevoice desktop application's main window is visible or not.
Files
On a new install the Program Files folder for babblevoice Desktop contains settings.lua and automation.lua. Settings contains generic settings which can be modified like the title or icon of the window. You can define all hotkeys in this file. The automation script (automation.lua) is the default automation script. If you want to customize automation it is recommended that you modify settings to point to your own file.
API Functions
----------------------------------------------------------------------
-- Hotkey: ^b
-- Purpose: Show/hide the main desktop window
----------------------------------------------------------------------
bv.registerhotkey("^b", function()
bv.showhide()
end )
All functions are placed in a global dictionary called bv, so to call showhide use bv.showhide().
alert
Alert the user with string. This will display the string in the toolbar icon balloon.
bv.alert("Info", "Hello World")
dial
Dial a number using bablevoice Desktop.
bv.dial("01442299280")
getforegroundwindow
This returns the handle of the foreground window. It is a wrapper to the Microsoft provided function. This is generally used to understand what application you are in, but also to pull information from.
local hwnd = bv.getforegroundwindow()
local windowtitle = bv.getwindowtext(hwnd)
-- windowtitle will now contain the string that appears in the Windows title bar of the application in the foreground. We can use this to see if it is the correct application, for example (Lua):
if nil ~= string.find(windowtitle, "EMIS") then
-- Simulate F5 pressed to bring up the search window.
bv.sendinput(bv.VK_F5, bv.KEYEVENTF_KEYDOWNUP, bv.INPUT_KEYBOARD)
-- We can then go onto populate the search field with the phone number of the call we are on.
end
setclipboardtext, getclipboardtext, waitforclipboardtext
All of these functions manipulate the clipboard. waitforclipboardtext takes the maximum number of mS to wait for text. You should set the clipboard text to "" before making this call as it will poll the clipboard until it has some text.
setclipboardtext("Hello World")
local text = getclipboardtext()
setclipboardtext("")
text = waitforclipboardtext(1000)
sendstring
Sends a string to the keyboard to emulate a user. It uses the MS API call SendInput to send text, but we attempt to make it simpler by taking a string representation of a string of characters to make it easier to send a sequence of keystrokes.
sendstring("^c\t^v")
The string can be any character on the keyboard (alpha numeric, space, tab (\t) and new line (\n)), some special control characters ( ^ control, + shift, ! Menu, ~ L Windows). For each character, a simulated key down followed by a key up will be sent. If a character is preceded by one of the special control characters, this is pressed followed by the keydown and keyup of the actual character, followed by the final release of the control character.
sendinput
This adds a simple wrapper to the underlying MS API SendInput call.
-- bv.sendinupt(key, hardware, action)
-- key is a constant, for keyboard these can be found at https://docs.microsoft.com/en-us/windows/desktop/inputdev/virtual-key-codes
-- hardware is either
-- * bv.INPUT_KEYBOARD
-- * bv.INPUT_MOUSE
-- action is either:
-- * bv.KEYEVENTF_KEYDOWN
-- * bv.KEYEVENTF_KEYUP
-- * bv.KEYEVENTF_KEYDOWNUP
-- When using mouse input, this should be used in conjunction with functions
-- like clienttoscreen to move the mouse to a specified location first.
bv.sendinput(bv.VK_F5, bv.INPUT_KEYBOARD, bv.KEYEVENTF_KEYDOWNUP)
registerhotkey
This is one of the core functions, it registers a system wide hotkey and the function to be called when pressed.
In the example, this is our default handler for when a user presses ^b (control-b)
--[===========================================[
Hotkey: ^b
Purpose: Show/hide the main desktop window
--]===========================================]
bv.registerhotkey("^b", function()
bv.showhide()
end )
getactivewindow
Gets the title text from the window which currently has focus.
getwindowthreadprocessid getcurrentthreadid attachthreadinput
getfocus
Returns the handle of the Window which currently has focus.
local hwnd = bv.getfocus()
setfocus
Calls the Windows function SetFocus. It returns the Window which previously had focus.
https://msdn.microsoft.com/en-us/library/windows/desktop/ms646312(v=vs.85).aspx.
getwindowtext
Returns the title text of the window.
local hwnd = bv.getfocus()
local text = bv.getwindowtext(hwnd)
getactivecontroltext, getcontroltext
Returns the text of in the text control which currently has focus.
getcontroltext is the same as getactivecontroltext but use a window handle to select the actual window.
getactivecontrolseltext
Returns the position of the text selection as a table.
local number = bv.getactivecontroltext()
local selection = bv.getactivecontrolseltext()
if selection.e > selection.s then
number = number:sub(selection.s, selection.e)
end
setactivecontroltext
Sets the text of the control which has focus.
showhide
Toggles a window show/hide.
showwindow
Calls the windows function ShowWindow. Constants are defined in the bv dictionary i.e. bv.SW_HIDE.
https://msdn.microsoft.com/en-us/library/windows/desktop/ms633548(v=vs.85).aspx
onevent
This function registers a callback which is called every time babblevoice Desktop receives an event. This includes information regarding phone calls, call queues and parking. Information can be passed from auto attendants and IVRs through the xinfo object.
onevent( function(ev)
if "device" == ev.type then
globvar = ev
end
end )
findwindow
This function maps directly onto the MS API call FindWindow.
enumchildwindows
This function maps directly onto the MS API call EnumChildWindow. It returns, as an array, all the child window handles.
windowfrompoint
This function maps directly onto the MS API call WindowFromPoint.
childwindowfrompoint
This function maps directly onto the MS API call ChildWindowFromPoint.
getclassname
This function maps directly onto the MS API call GetClassName and returns the class name as a string.
setcursorpos, getcursorpos, screentoclient, clienttoscreen, mapwindowpoints
All of these functions either get or set or manipulate mouse cursor positions.
-- A point is a table, ie. point = { x: 100, y: 200 }
setcursorpos(point)
local point = getcursorpos()
point = screentoclient(hwnd, point)
point = clienttoscreen(hwnd, point)
point = mapwindowpoints(hwnd1, hwnd2, point)
-- Example:
local hwnd = bv.getforegroundwindow()
-- We are interested in a child window - not the main parent
local childs = bv.enumchildwindows(hwnd)
local p = { x = 50, y = 60 }
-- We want to press buttons relative to the first child window.
local ptowin = bv.clienttoscreen(childs[1], p)
-- Now press some buttons.
bv.sendinput(bv.VK_LBUTTON, bv.INPUT_MOUSE, ptowin)
bv.sleep(50)
local sub = { y = ptowin.y + 75, x = ptowin.x + 400 }
bv.sendinput(bv.VK_RBUTTON, bv.INPUT_MOUSE, sub)
sleep
Sleep for a number of mS. Maximum 10000mS.
gethtmlfromieserver
Returns a text string of the HTML in a 'Internet Explorer_Server' control.
local hwnd = bv.getforegroundwindow()
local childs = bv.enumchildwindows(hwnd)
for iterator = 1, #childs do
local c = bv.getclassname(childs[iterator])
if "Internet Explorer_Server" == c then
local html = bv.gethtmlfromieserver(childs[iterator])
local file = io.open("mytempfile.txt", "w")
file:write(html)
file:close()
end
end
getwindowrect
Returns a rect dictionary of the window in question.
local hwnd = bv.getforegroundwindow()
local rect = bv.getwindowrect(hwnd)
-- rect is either nil or now contains, top, bottom, left and right.
