-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathbot.html
More file actions
39 lines (35 loc) · 1.39 KB
/
bot.html
File metadata and controls
39 lines (35 loc) · 1.39 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<title>BrawlBot</title>
<script src="sharedHeader.js"></script>
</head>
<body>
<nav-bar></nav-bar>
<br />
<div class="container my-4">
<p>
BrawlBot Short for Bread's Crawl Bot is a web crawler being developed by
yours truely. I am experimenting with writting a search engine! It is
still under development. If you would like to block the crawler from
your page, it will respect <code>robots.txt</code> as defined here:
<a href="https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec">
developers.google.com/crawling/docs/robots-txt/robots-txt-spec
</a>. As of publishing this page (Sat May 30 2026)
<code>robots.txt</code> does not work. This page will be updated as
changes are made.
</p>
</div>
<footer-links></footer-links>
<br />
<script type="module" src="navBar.js"></script>
<script type="module" src="themeListener.js"></script>
<script type="module" src="Accordion.js"></script>
<script type="module" src="footer.js"></script>
<script type="module" src="aOpenNew.js"></script>
<script type="module" src="ResumeSection.js"></script>
</body>
</html>