Exposed APIs: automating web pages without the GUI

Exposed APIs: automating web pages without the GUI

The problem

Automating some actions on a web page is a very common use case for Robotic Process Automation (RPA). Whether it's scraping some data, or filling in some form, RPA works wonders, because it can mimic the user via the general user interface. This is great for those external web pages where a direct integration is not an option due to a lack of a proper application programming interface (API) and the fact it's external (i.e. no direct access to the database behind it).

The problem with doing that, however, is that web page GUIs change frequently. Some of those changes may not even be visible to the human eye, but even a minor change in the underlying page structure can break the flows intended to automate it.

Even worse still, is the fact that most external web pages will get these updates without any prior warning to the users. So, there will be no time to prepare for the changes needed to make the flows work again. It will stop working without any warning and then someone will need break quite a bit of sweat to get it back to being operational again.

But what if I told you there's a better way to do it than mimicking the same actions a human would perform on the GUI?

The solution - exposed APIs

The fact a web page does not provide an API service does not necessarily mean there is no API to tap into. Quite a few web pages out there actually use an API to fetch data from their servers or to push data back to their servers.

I call these APIs "exposed APIs" mainly because they are undocumented (i.e. the service provider does not actually provide the API as a service to the general public), but are still available. They are exposed to the public and using them is usually not against any kind of a license agreement. These APIs can easily be found and reverse-engineered to be used in your automation.

This results in:

  • Higher efficiency - no need to use a browser at all, much less requests are likely needed and they usually work faster, due to connecting directly to the server without using the web page as an intermediary;
  • Greater stability - a server API, even if it is undocumented for the public, will change much less frequently than the GUI of the web page, meaning less failures and thus less support and maintenance*;
  • Lower cost - it is usually easier to build flows like that, meaning development cost is lower, not to mention lower support and maintenance cost due to, well, less support and maintenance needed.

* Note: I should mention that these APIs are not really targeted at the general public. They are not documented and if they do change, anyone using them will not be notified, nor will they find any resources on what exactly changed. In a case like that, one would need to reverse-engineer the API again. But as long as I've been using them for the past few years, I have not encountered a case like that.

And since I'm a Microsoft Power Automate fanboy, I will also obviously mention that when it is no longer required to automate a web page GUI to get what I need, I can easily build the same flow in Power Automate cloud without the need to run it as a desktop flow in Power Automate Desktop. This means I can do the same with a lower license cost and without any extra IT infrastructure.

An example

To better illustrate the idea of using exposed APIs, I'll describe an actual use case example. It's a very simple one that only requires a single web service request. I specifically picked one like this to not make the article too long (it's already long as it is).

Context

The following example was inspired by a user asking for help in the I Love Automation Discord server. The user was trying to scrape the events on The Poisoned Pen Bookstore as listed on their event calendar. But they were having trouble retrieving all events and then clicking on each of them to get the details via the GUI.

Finding the API

I came to help, and lo and behold - I quickly found that the page has an exposed API we could tap into. Upon opening the developer tools in the browser (Chrome in my case), navigating to the Network tab, filtering for events of type Fetch/XHR and reloading the page, I found a few that were available:

No alt text provided for this image
The Network events after loading the page

The two 'collect' requests are just Google Analytics plug-ins sending data of us browsing the page. But the first one (the one that starts with 'events') is an actual HTTP request to the server that returns the data on the events.

Building the flow to send the request

I looked up the Headers section of the specific request and could easily mimic this in PAD:

No alt text provided for this image
The request headers

It also has some parameters passed into the URL, as it does not actually require any request body.

The parameters are as follows:

  • start_date_utc
  • per_page
  • page

The per_page and page parameters are quite self-explanatory. We can set the number of items to be returned in a single page with the per_page parameter and then use page for pagination. I've tested it with per_page=1000 and it did not error out. But I could not confirm whether it would return 1000 events, as there were only 58 in total available. But this at least proves that more than 30 can be retrieved in a single page.

The one that is a bit trickier here is the start_date_utc parameter which actually requires the so-called Unix Time Stamp in order to be used as a date for filtering the relevant events.

For more info on what the Unix Time Stamp is and how to calculate it, see the Epoch Converter. But the general idea is that it is the difference between the date in question and 01-01-1970 in seconds or milliseconds.

I've tested this and also received a response saying that the date must be an integer, implying that the use of current date and time is not permitted. So, it should be the difference between the current date at 00:00:00 and 01-01-1970 at 00:00:00 in seconds. It's quite simple to calculate in PAD by using the following three actions:

No alt text provided for this image
PAD actions to calculate the Unix Time Stamp

It's not really visible in the screenshot, but the Get current date and time action is set to only retrieve the date and ignore the current time. For some quicker access, here's the Robin code behind these actions (this and any further snippets are made with PAD version 2.31.105.23101):

DateTime.GetCurrentDateTime.Local DateTimeFormat: DateTime.DateTimeFormat.DateOnly CurrentDateTime=> CurrentDateTime
Text.ConvertTextToDateTime.ToDateTimeCustomFormat Text: $'''01-01-1970''' CustomFormat: $'''dd-MM-yyyy''' DateTime=> TextAsDateTime
DateTime.Subtract FromDate: CurrentDateTime SubstractDate: TextAsDateTime TimeUnit: DateTime.DifferenceTimeUnit.Seconds TimeDifference=> TimeDifference        

With that, we have the necessary parameters to build our URL for the endpoint, so we can start building our request. To do that, we will use the Invoke web service action under the HTTP actions group:

No alt text provided for this image
The Invoke web service action in PAD

When added to the flow, it looks like this and we need to provide some parameters to it:

No alt text provided for this image
An empty Invoke web service action

Now, there is no body needed, but we do need some headers for the request.

The screenshot of the network traffic for the page contains quite a few request headers, but most of them are in fact irrelevant or are handled automatically by PAD. What does matter are the following:

  • method
  • accept
  • x-api-key

The method header is used to define what the request is supposed to do. In most cases, when only fetching data with a blank body, a GET request is used. This is the case here, too. We generally don't need to think much about the method - we simply copy what we see in the sample request headers and put it into the Method field (actually a drop-down list) in PAD:

No alt text provided for this image
The Method field in PAD

The accept header is used to tell your app (in this case PAD) what sort of contents it should expect to be returned in the response. In this case we get a JSON formatted response and thus it is quite normal to have application/json in the accept header. This also has a pre-built field in PAD:

No alt text provided for this image
The Accept field in PAD

I would normally also set the Content type header value to define the data type of the request body. But since there is no body needed for this request, we can simply leave it as it is by default, or put in */* (meaning any content type is valid):

No alt text provided for this image
The Content type field in PAD

Finally, there's the x-api-key header, which is a bit more custom. It seems like the web page actually has an API key that it uses to access the server and fetch info. In some cases, there may be a Cookie instead that may need to be generated by running a separate request first and then parsing its response.

But in this case it is actually a static value. I tested it over different browsers and different dates and the API key doesn't change. Seems like it is issued by the TimelyApp to the Poisoned Pen Bookstore to fetch their events only.

So, all we need to do is copy the API key to the Custom headers field in PAD:

No alt text provided for this image
The Custom headers field in PAD

Note: API keys that are issued to you specifically by a service provider for private access to your data should be stored securely and you should never share them with anyone. I'm only sharing this one because it is publicly available anyway and it's not private data. If it was a private key, I would store it in a password vault and get it dynamically via the flow, instead of hard-coding it into the Invoke web service action.

With all that, we end up with an action like this:

Web.InvokeWebService.InvokeWebService Url: $'''https://timelyapp.time.ly/api/calendars/54705163/events?start_date_utc=%TimeDifference%&per_page=1000&page=1''' Method: Web.Method.Get Accept: $'''application/json''' ContentType: $'''*/*''' CustomHeaders: $'''x-api-key: c6e5e0363b5925b28552de8805464c66f25ba0ce''' ConnectionTimeout: 30 FollowRedirection: True ClearCookies: False FailOnErrorStatus: False EncodeRequestBody: True UserAgent: $'''Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.21) Gecko/20100312 Firefox/3.6''' Encoding: Web.Encoding.AutoDetect AcceptUntrustedCertificates: False ResponseHeaders=> WebServiceResponseHeaders Response=> WebServiceResponse StatusCode=> StatusCode        

As you can see, I provide the %TimeDifference% value into the URL as a variable, but leave everything else hard-coded. If you wanted to do pagination (or if the page did not allow passing in 1000 into the per_page parameter), you might want to also build a loop that generates a page index and then pass that as a variable next to the page parameter. It was not necessary here, so I left it as it is.

Parsing the response

When I run the flow, the Invoke web service action generates several variables. The WebServiceResponseHeaders are sometimes useful to get a cookie if it was needed. The StatusCode returns an HTTP status code for the request (in general 200 means success, but for a complete list see here). It can also be used for some error handling to verify your request succeeded before trying to use the response data.

The most important piece here is the WebServiceResponse variable which contains the data we need. As mentioned, it's a JSON blob. It looks somewhat like this (with the actual items being collapsed except for the first one to save some space in the article):

{
? ? "data": {
? ? ? ? "total": 58,
? ? ? ? "from": 0,
? ? ? ? "size": 1000,
? ? ? ? "has_prior": false,
? ? ? ? "has_next": false,
? ? ? ? "items": [
? ? ? ? ? ? {
? ? ? ? ? ? ? ? "ticket_type": "no_ticket",
? ? ? ? ? ? ? ? "cost_display": "0",
? ? ? ? ? ? ? ? "notify_subscribers": false,
? ? ? ? ? ? ? ? "featured": false,
? ? ? ? ? ? ? ? "instance": "20230430170000",
? ? ? ? ? ? ? ? "start_datetime": "2023-04-30 17:00:00",
? ? ? ? ? ? ? ? "end_datetime": "2023-04-30 18:00:00",
? ? ? ? ? ? ? ? "cost_type": null,
? ? ? ? ? ? ? ? "timezone": "America\/Phoenix",
? ? ? ? ? ? ? ? "end_utc_datetime": "2023-05-01 01:00:00",
? ? ? ? ? ? ? ? "instant_event": false,
? ? ? ? ? ? ? ? "created_at": "2023-03-09 02:37:12",
? ? ? ? ? ? ? ? "post_to_facebook": false,
? ? ? ? ? ? ? ? "title": "VIRTUAL EVENT: ?Dennis Lehane with Michael Koryta",
? ? ? ? ? ? ? ? "is_example_event": false,
? ? ? ? ? ? ? ? "feed_id": null,
? ? ? ? ? ? ? ? "uid": "4bc1a120-be23-11ed-9aa7-816ece8f5911-time.ly",
? ? ? ? ? ? ? ? "start_utc_datetime": "2023-05-01 00:00:00",
? ? ? ? ? ? ? ? "updated_at": "2023-03-31 22:00:40",
? ? ? ? ? ? ? ? "post_to_linkedin": false,
? ? ? ? ? ? ? ? "id": 76057132,
? ? ? ? ? ? ? ? "event_status": "confirmed",
? ? ? ? ? ? ? ? "custom_url": "virtual-event-52",
? ? ? ? ? ? ? ? "tickets_count": 0,
? ? ? ? ? ? ? ? "images": [
? ? ? ? ? ? ? ? ? ? {
? ? ? ? ? ? ? ? ? ? ? ? "small": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 180,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq_small._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 272
? ? ? ? ? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? ? ? ? ? "thumbnail": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 70,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq_thumbnail._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 106
? ? ? ? ? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? ? ? ? ? "reference_id": 54705163,
? ? ? ? ? ? ? ? ? ? ? ? "file_name": "814RsaMtNuL_WIQq._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? "created_at": "2023-03-09 02:49:50",
? ? ? ? ? ? ? ? ? ? ? ? "medium": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 500,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq_medium._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 755
? ? ? ? ? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? ? ? ? ? "title": "814RsaMtNuL._SL1500_.jpg",
? ? ? ? ? ? ? ? ? ? ? ? "deleted_at": null,
? ? ? ? ? ? ? ? ? ? ? ? "file_size": 223017,
? ? ? ? ? ? ? ? ? ? ? ? "sizes": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "small": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 180,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq_small._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 272
? ? ? ? ? ? ? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? ? ? ? ? ? ? "thumbnail": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 70,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq_thumbnail._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 106
? ? ? ? ? ? ? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? ? ? ? ? ? ? "medium": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 500,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq_medium._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 755
? ? ? ? ? ? ? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? ? ? ? ? ? ? "full": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 994,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 1500
? ? ? ? ? ? ? ? ? ? ? ? ? ? }
? ? ? ? ? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? ? ? ? ? "updated_at": "2023-03-09 02:49:50",
? ? ? ? ? ? ? ? ? ? ? ? "file_hash": "4a4b3d336f7e821b6fbc75d7f4d07f6a",
? ? ? ? ? ? ? ? ? ? ? ? "mime_type": "image\/jpeg",
? ? ? ? ? ? ? ? ? ? ? ? "user_id": 30512,
? ? ? ? ? ? ? ? ? ? ? ? "id": 678504157,
? ? ? ? ? ? ? ? ? ? ? ? "full": {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "width": 994,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "url": "https:\/\/timelyapp-prod.s3.us-west-2.amazonaws.com\/images\/54705163\/814RsaMtNuL_WIQq._SL1500_",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "height": 1500
? ? ? ? ? ? ? ? ? ? ? ? }
? ? ? ? ? ? ? ? ? ? }
? ? ? ? ? ? ? ? ],
? ? ? ? ? ? ? ? "cost": null,
? ? ? ? ? ? ? ? "cost_external_url": null,
? ? ? ? ? ? ? ? "calendar_id": 54705163,
? ? ? ? ? ? ? ? "post_to_twitter": false,
? ? ? ? ? ? ? ? "tickets_min_price": null,
? ? ? ? ? ? ? ? "tickets_currency_symbol": null,
? ? ? ? ? ? ? ? "tickets_currency": null,
? ? ? ? ? ? ? ? "allday": false,
? ? ? ? ? ? ? ? "description_short": "Dennis Lehane discusses \nSmall Mercies\nclick here to order! \n\u00a0\nVery special guest host: Michael Koryta\n\u00a0\nDennis Lehane. Small Mercies (Harper Collins, $29.99 Signed).\nThe acclaimed New York Times best…",
? ? ? ? ? ? ? ? "taxonomies": {
? ? ? ? ? ? ? ? ? ? "taxonomy_tag": [
? ? ? ? ? ? ? ? ? ? ? ? {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "image": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "reference_id": 54705163,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "color": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "updated_at": "2021-04-02 22:21:49",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "item_type": "taxonomy_tag",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "created_at": "2021-04-02 22:21:49",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "id": 677471322,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "image_id": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "title": "Signed books",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "deleted_at": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "filter_group_id": null
? ? ? ? ? ? ? ? ? ? ? ? }
? ? ? ? ? ? ? ? ? ? ],
? ? ? ? ? ? ? ? ? ? "taxonomy_venue": [
? ? ? ? ? ? ? ? ? ? ? ? {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "country": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "image": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "website": "https:\/\/www.youtube.com\/channel\/UCTbRuLNmD8EhT4WrGGMVW-w",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "address": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "geo_location": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "reference_id": 54705163,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "address2": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "city": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "item_type": "taxonomy_venue",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "phone2": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "created_at": "2021-11-10 23:46:31",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "title": "Facebook Live and Youtube Live",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "deleted_at": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "country_first_division": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "updated_at": "2021-11-10 23:46:31",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "phone": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "venue_type": "O",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "id": 678162934,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "image_id": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "postal_code": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "email": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "place_id": null
? ? ? ? ? ? ? ? ? ? ? ? }
? ? ? ? ? ? ? ? ? ? ],
? ? ? ? ? ? ? ? ? ? "taxonomy_category": [
? ? ? ? ? ? ? ? ? ? ? ? {
? ? ? ? ? ? ? ? ? ? ? ? ? ? "image": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "reference_id": 54705163,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "color": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "updated_at": "2021-04-02 22:24:00",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "item_type": "taxonomy_category",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "created_at": "2021-04-02 22:04:35",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "id": 677471320,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "image_id": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "title": "Virtual event",
? ? ? ? ? ? ? ? ? ? ? ? ? ? "deleted_at": null,
? ? ? ? ? ? ? ? ? ? ? ? ? ? "filter_group_id": null
? ? ? ? ? ? ? ? ? ? ? ? }
? ? ? ? ? ? ? ? ? ? ]
? ? ? ? ? ? ? ? },
? ? ? ? ? ? ? ? "user": "Patrick Millikin ([email protected])",
? ? ? ? ? ? ? ? "url": "https:\/\/calendar.time.ly\/9plshfqx\/event\/76057132\/20230430170000",
? ? ? ? ? ? ? ? "canonical_url": "https:\/\/calendar.time.ly\/9plshfqx\/event\/76057132"
? ? ? ? ? ? },
? ? ? ? ? ? <...>
? ? ? ? ]
? ? }
}        

As you can see, it is an object that contains a single data object inside it, which has some header values (including the total number of results) and the items node that contains a further list of objects. These are the actual events.

So, the next step is to convert it to an object that PAD can parse. We do that by using the Convert JSON to custom object action.

Variables.ConvertJsonToCustomObject Json: WebServiceResponse CustomObject=> JsonAsCustomObject        

When we do that, we can easily access the values inside the object. We can do that by using the proper variable notation. A property of an object can be accessed in two different ways:

  • %Object.Property%
  • %Object['Property']%

I prefer the second notation, as it works with variables being passed as the property, too (variables should be passed without the quotation marks as %Object[Property]%).

So, the following Set variable action will get me the total number of events:

SET TotalEvents TO JsonAsCustomObject['data']['total']        

And then we can do a For each loop on the items in the object to process each event individually by doing:

LOOP FOREACH CurrentItem IN JsonAsCustomObject['data']['items']
  # do something here
END        

The data can then be written to an Excel file, pushed to SharePoint or a database, sent via email as a notification, etc. You can do whatever you need to do it. The important thing here is that this allows skipping the use of a browser and automating a web UI that can change frequently.

Cloud flows

This actually means that the entire flow can be built in cloud flows and desktop flows are not even needed (assuming the output data does not need to go into some other UI). I described this example using code from a desktop flow, as it can be done in less steps. But the same (using slightly different actions) can also be done in cloud flows.

Doing so would actually make the flow more efficient and faster. Cloud flows support parallel execution, unlike desktop flows. It also allows scheduling and triggering the flow without the need to have the license that includes desktop flows, too. Also, it would not require a machine to run on.

But it would require a paid license of Power Automate, as the HTTP actions needed to send a request to the API endpoint are considered Premium. On the other hand, Power Automate Desktop flows, when launched manually as attended flows, could be run for free.

For the sake of not making this article too long, I will skip describing the sample in Power Automate cloud flows this time. But if the readers show interest, I will describe that in a separate article.

TL;DR

Using APIs that the web page uses to retrieve data from its servers is a way to make flows more efficient and robust. Flows like that require less maintenance as APIs change less frequently than UIs. It also makes the flows much faster and require less steps.

Using them is usually quite simple and straight-forward if you know what you're doing. This article provides a step-by-step guide on how to find an existing exposed (undocumented) API on a web-page, how to reverse-engineer it and then implement these findings in a Power Automate Desktop flow.

About me

I specialize in helping businesses improve and automate their processes. If you see any value in what I shared here, please follow me for more insights into , , and similar topics. Hit the ?? on my profile to get a notification for all my new posts. ??

If you need any help with making your business more efficient, contact me or my team at Definra ??

Jesper Stephansen

Power Platform & AI ekspert - Futurist - Power Automate - RPA Teacher - Copilot - Copilot 365 - Copilot Studio - ChatGPT

1 年

This is great. thank you for sharing!

Emre ?ZGüRüO?LU

Co-Founder | Software Development Manager

1 年

We are already doing it with GENRPA on Turkey. Also you can manage GraphQL files of websites like this. Very informative for those interested in RPA. Thank you so much. ??

Gerrit Jan Hagens

Partner & CTO @ Shared | Aanjagers van vooruitgang ?? Boosting progression | (Low) Code | IPaaS | RPA | AI | Cloud | Power Platform

1 年

You're right, we always prefer this method above RPA due to cost, ease of development and run time performance. However, in case of more advanced web apps, it sometimes can be a pain to build all the logic in Power Automate cloud, for which one for example would require RegEx. The only exception when this API-method doesn't apply is in the case of additional execution of frontend JavaScript, for which a browser is required. Even then, headless scraping using an Azure Function often pays off. Ergo: this isn't for Citizen developers.

Jason Savory

Senior Automation Developer and Freelance Media Composer

1 年

I’ve taken a slightly different angle in the past by injecting JS to execute button clicks, saves me from using selectors and is more resilient to change.

要查看或添加评论,请登录

Agnius Bartninkas的更多文章

社区洞察

其他会员也浏览了