Computer Use: Letting the AI Use the Screen

1. About this article

Computer use means that the AI agent looks at images of your screen and uses the mouse and keyboard in your place. The agent opens menus, clicks buttons, types values into windows and checks the result, again with a screenshot.

When is it used?	Why
If your StatiCAD edition has no terminal channel (Profesyonel, Standart, Betonarme, Yığma)	The agent can operate the program only through the screen.
For work that has no terminal command	Such as settings in dialog windows, selecting, moving, views and the report preview.
For visual checks	So that the agent checks the model it has built in the plan and 3D views with its own eyes.

info

If there is a terminal, the terminal first: Computer use is much slower than terminal commands and can make clicking mistakes. In Ultimate and Hybrid, the agent should use terminal commands first and switch to the screen only when needed. The first prompt in the Claude Code article already says this.

2. With which tools?

Tool	Status on Windows	Plan required
Claude desktop app (Claude Code)	Available (research preview).	Claude Pro or Max
OpenAI Codex	Available; runs in the foreground on Windows. Not available in the EEA, the United Kingdom and Switzerland.	ChatGPT plan
Google Antigravity	Not documented for desktop programs.	—

3. Prepare the computer

1. Set the display scale to 100% and, if possible, the resolution to 1920×1080 (Windows Settings › System › Display). The StatiCAD User Guide and AGENTS.md were prepared for this setting; larger scales increase clicking errors.
2. Open StatiCAD and maximize its window so that it fills the screen; the hardware key should be plugged in.
3. Close unnecessary windows. In particular, close windows that show personal information, such as email, banking and password managers.
4. Temporarily silence Windows notifications (Focus / Do not disturb); notifications that pop up can confuse the clicks.

4. Turning on computer use in Claude

1. Update the Claude desktop app (if Claude does not update itself, reinstall it from claude.com/download).
2. In Settings, in the desktop app section (Desktop app), turn on the Computer use toggle. On Windows, the setting takes effect immediately.
3. Start your session in the Code tab (Claude Code, Step 6).
4. The first time Claude is about to use the screen, it asks which applications it wants to use; allow StatiCAD. It asks separately for other applications when needed.

info

Permission levels: Claude can fully use the applications you allow. For security, it can only view web browsers, and in terminals and code editors it can only click (it cannot type). The applications you allow are limited to that session.

5. Computer use in Codex

• Turn on computer use in the settings of the Codex app.
• On Windows, windows come to the foreground while Codex is using the screen; do not touch the mouse and keyboard during that time.
• Installation and account: ChatGPT (Codex) article.

6. Prompt examples

Ultimate and Hybrid: terminal + the screen when needed. The prompt in the Claude Code article is enough:

Read the AGENTS.md and YARDIM_AI_KOMUTLAR.md files in the C:\StatiCAD\Ultimatex64 folder. I want you to create my projects by using the program (StatiCAD.exe) through the terminal and, when needed, with computer use.

Editions without a terminal (Profesyonel as an example; change the folder name to suit your edition: V10StdX64, BAV10, YigmaV10ProX64, YigmaV10StdX64; in Betonarme the program file is StatiCAD-Betonarme.exe, in Yığma StatiCAD-Yigma.exe):

I am using StatiCAD Profesyonel; the program file is C:\StatiCAD\V10ProX64\StatiCAD.exe. This edition has no terminal channel; you will use the program through the screen with computer use.
- First read AGENTS.md if it exists in the C:\StatiCAD\V10ProX64 folder; if it does not, proceed by reading the screen.
- If the program is open, do not start it a second time. Before every click, verify the target with a screenshot.
- Ask me before overwriting existing files and before closing the program.
When you are ready, take a screenshot of the main screen, briefly describe what you see and ask me what I want to do.

info

Agent instruction files: The instruction files for agents (AGENTS.md and a working guide specific to computer use: recognizing the screen state, reliable clicking, waiting points, common mistakes) are only included in the Ultimate installation. In other editions, the prompt above itself gives the agent the most essential rules from these files.

In the middle of a session, to direct the agent to the screen:

Use computer use for work that cannot be done with a terminal command: first take a screenshot of the StatiCAD window on the screen, then open the relevant menu or window. If a dialog stays open, the terminal commands wait; check the dialog and close it. At the end of the job, take a screenshot of the 3D view and give me a summary confirming that you have checked the model.

7. While the agent is working

• Do not touch the mouse and keyboard; the agent's clicks get mixed up with yours.
• Follow what the agent is doing in the chat window. If it goes to a wrong window or menu, interrupt the job with the stop button and write again what it should do.
• If an unwanted change has happened in StatiCAD, undo it with Ctrl+Z after stopping.
• Ask the agent to take a screenshot at important steps (e.g. when the analysis finishes, in the report preview) and summarize what it sees.

8. Security

danger

You are letting someone else use your screen:

• The agent can only use the applications you allow; before allowing applications other than StatiCAD, ask why.
• Do not have the agent type passwords, card details or ID numbers; if you see such a request, stop the job.
• Everything visible on the screen (open documents, notifications) goes into the image sent to the AI company. Close confidential information.
• If the agent sees a licence warning such as “Kilit Bulunamadı” (Lock Not Found), it should stop and inform you; handle the hardware key and licence yourself.

9. Troubleshooting

Symptom	Cause and solution
The agent clicks in the wrong places	The display scale is not 100% or the window is small. Set the scale to 100% and maximize StatiCAD; ask the agent to verify with a screenshot before every click.
The computer use option is not visible	The app is not up to date or the plan is not suitable (Claude requires Pro/Max). Update and try again.
The agent cannot find the menus	Tell it the menu names yourself (e.g. “RUN STATIC ANALYSIS from the Analysis menu”) or ask it to read AGENTS.md (in the Ultimate folder).
The agent got stuck in a window	A question or warning window may have been left open. Tell it to check the window, or close it yourself.
The computer cannot be used while Codex is working	On Windows, Codex runs in the foreground; wait until the job is finished.

The plan, price and menu names of Anthropic, OpenAI and Google services may change over time; for up-to-date information, see the linked official pages. The model built and the results produced by the AI must be checked by the responsible engineer.