2

For a long time, the EL&U community has had a problem. We have been unable to search for stop words (common words that are not searchable), which is "problematic since many of the questions on the site [are] about stop words."

Six months ago, the SE search engine introduced a feature that, if conscientiously implemented, would give us a workaround for not being able to search on stop words.

That feature is the ability to search for any text contained within code tags.

Theoretically, the code tags could therefore be used to enclose stop words that should not be ignored by the search engine.

Now, we have discouraged the use of code tags on the site, in part because they look ugly, and in part because blockquotes are really more suited to what we do here.

However, the argument has been made that keywords, i.e. the words we use to find the particular posts we are looking for, are essentially the "code" of our site. In other words, we could use code tags around words in the posts (particularly around relevant stop words) in order to improve the efficiency of searching because stop words enclosed in code tags would no longer be ignored.

For example, right now, you cannot easily find the question "What is the best way to explain how to choose between “its” and “it's”?" using the site search engine because "its" and "it's" are stopwords.

If we insert code tags in the body—What is the best way to explain how to choose between its and it's?—the question becomes much easier to find because we can look for "its" directly.

So if we consider marking keywords with code tags, currently keywords would be formatted thus. There exists the possibility that we can alter this formatting to something more appealing and site-specific. The question is

What is an acceptable, obvious, and standard method of marking keywords?

Some suggestions have been

  • Italics
  • Italics plus underline
  • Italics plus bold
  • Italics plus dotted underline
  • Small caps
  • Small cap bold italics plus dotted underline plus green plus blink
  • Leave it like it already is (and perhaps stick them at the bottom)
  • Make them look like regular text

Text would be tagged like it is now, by the poster or the community as we decide that it is needed for the purposes of making the site more searchable. It might have helped this user this morning, for instance.

What says the community?

  • 4
    I am leaning towards green blink. But would it be conspicuous enough without a moving pop-up? – Cerberus_Reinstate_Monica Sep 10 '12 at 20:10
  • 4
    I'm confused by what you mean by "keywords". – nohat Sep 10 '12 at 20:55
  • 1
    1. Whose job will it be to decide what the keywords are? Is this something the mods will do, or is it up to the poster, or does the whole community get to kick in? 2. Are there alternatives to disfiguring the copy? Could we stick a keywords line at the bottom? Labeled something cheery like "Keywords:", or not. Then it wouldn't really matter what you did with them. – StoneyB Sep 10 '12 at 22:04
  • 2
    I don't understand this. What are "keywords", where/why would I find/use them, and why does it matter how they're displayed? – FumbleFingers Sep 10 '12 at 22:35
  • @Fumble The words we use to find the things we are looking for. This is problematic on this site because the search ignores stop words (like "its") which are things we might actually want to search for here; you might not ever use them if you don't search for stuff; because some people don't like the way code tags look and they leave the impression that we are talking about code when we're not. – Kit Z. Fox Sep 10 '12 at 22:43
  • 1
    @KitFox: Seems like an odd usage to me. Since the site indexer ignores "stop words" like its, I don't bother with them in my search terms. Unlike Google, SO searches don't seem to honour "quotated search strings", so I still don't see what you're getting at. – FumbleFingers Sep 10 '12 at 22:59
  • @Fumble The fact that the site indexer ignores stop words is entirely the point, and simply omitting them when one is searching for a question on the difference between "its" and "it's" is not going to yield very useful results. – Kit Z. Fox Sep 10 '12 at 23:57
  • 1
    I must be extra thick today. I know they don't work as search terms - but I don't know what you're proposing to do about that, or why it currently makes any difference how the site displays words that are no use to us anyway. No use at the moment, at least, and I don't see how that can change unless TPTB give us our own site-specific index & search facilities. – FumbleFingers Sep 11 '12 at 0:06
  • @Fumble Why do you think "its" versus "it's" is no use to us anyway? What about when someone is looking for a dupe? – Kit Z. Fox Sep 11 '12 at 0:15
  • @FumbleFingers, if I understand it correctly, TPTB are giving us (or rather, have given us) a tool that we can use to index our own content, and that indexing then appears in search results. So, it's site-specific index & search facilities, but via a workaround of sorts. – Marthaª Sep 11 '12 at 0:28
  • 2
    I for one still do not understand what you mean by keyword. Are you talking about displaying things in search results? Or are you talking about putting markup around stop words in the original text? or around non stop words in the original text? Or what? Presumably, we don't want the original text of questions and answers to look any different at all, whether stop or non stop words. Or maybe you're saying we do? You want -all- stop words to be formatted differently? " the man on the bike " ? We're just trying to create a hack that will allow us to search on stop words? And look crappy? – Mitch Sep 11 '12 at 1:16
  • @Mitch: you'd only mark a stop word as a keyword if it's, well, a key word, e.g. if you're talking about the pronunciation of the. – Marthaª Sep 11 '12 at 1:31
  • And the point is to make it not look crappy. – Kit Z. Fox Sep 11 '12 at 1:36
  • 2
    Some background: we have had a search parameter code: silently implemented for a while. While its purpose and intent remains for our programming oriented sites to help search for otherwise undiscoverable elements such as ++ and -->, I realized it could also avoid the usual issue of stop words (among other curations possible). So I, personally, stepped over here and brought word of it as a potential option to use, if people were interested in using the syntax. That's really the gist of it from the SE end - just one person who thought it might be nice if people liked it. – Grace Note Sep 11 '12 at 17:18
  • 1
    @GraceNote What words are exactly considered stop words, on Stack Exchange? – kiamlaluno Sep 11 '12 at 22:30
3

If I understand this correctly

  • The SE search engine ignores a finite number of words—stop words—such as articles, particles, perhaps prepositions and copulae and so forth
  • But in many cases these stop words themselves are the subject of our posts, and are precisely the key words an ELU user might employ in a search
  • A cheap and easy workaround is to mark any stop words which would serve as appropriate key words for the post with code tags, which override the 'ignore' directive and include the marked stop words in the search
  • There is no need to mark any word which is not a stop word
  • There is no need to mark any stop word more than once in any post

If these are all true, then the issues before the community are

  1. Where shall stop words be marked with code tags be entered?—in the body of the post, or at some place apart, such as at the end?
  2. Who shall be responsible for marking appropriate stop words?
  3. How shall marked stop words be displayed?—with the existing formatting or with some other formatting peculiar to ELU?

My votes on these issues:

  1. To avoid confusing the reader, marked words should be entered at the end of the post, with the label Andrew Leach offers, as below (with or without the blockquoting)
  2. The poster should take responsibility for entering appropriate Additional Keywords; but like any other part of the post, they may be added to or subtracted from by community editors.
  3. If the marked words aren't in the body, who cares? Leave it as it is, and don't create work for anybody.

Additional Keyword: KISS

  • Yes, I was thinking we could add the words ad hoc to existing posts to make them easier to find. – Kit Z. Fox Sep 11 '12 at 11:00
  • Yes. If the keywords are going to be in a block at the bottom, who cares? If you're talking about using <code> inline in posts to signal keywords, that's a different matter. – Monica Cellio Sep 11 '12 at 15:58
  • I'd be happy with this, because it's an obvious way of using the additional search functionality without upsetting anything else (including tables, but the possible exception of RegDwight) and it means we can carry on squashing inline code in the text, which is ugly and unnecessary. I like the blockquote, too; it's rather like an extra row of tags. – Andrew Leach Sep 11 '12 at 19:13
  • However I do think that the grey background is Not A Good Thing. The background for code should be the same colour as tags currently are -- both because it looks better and keywords are also meta-data. – Andrew Leach Sep 11 '12 at 19:54
  • Except the way I see this being most useful is if you choose backticks to make a use/mention distinction, it will automagically cause your keywords to show up in search results -- no extra work required. And for that to be a viable option, the formatting needs to be Not Ugly. – Marthaª Sep 12 '12 at 19:01
  • @Martha I find any visible mark in the body copy which is not a part of my argument to be gratuitous, intrusive and distracting, regardless of whether it's ugly. I'd rather take three seconds to post the words at the end. – StoneyB Sep 12 '12 at 19:25
  • @StoneyB: if you're talking about words, rather than using them, then you kind of have to mark the words somehow, don't you? Compare "Do you need a that in the following sentence?" to "Do you need a that in the following sentence?" The former sounds nonsensical; the latter, in addition to clarifying your meaning, makes your question searchable even though the main point is a stop word, without needing to add more text at the bottom. The problem is, it's ugly. This is why we need to change the code formatting. – Marthaª Sep 12 '12 at 19:53
  • @Martha Suppose I need to mark "that" "yon" "haggis" and "Scots" as terms under discussion -- why should I be constrained either to use a different marker for "that" than for the others or to mark them all as 'code'? What am I telling my reader? If I enter "that" explicitly as a keyword at the end, I'm free to use whatever marker seems appropriate in the body. – StoneyB Sep 12 '12 at 20:01
  • @StoneyB: nobody's preventing you from adding a keyword section at the end, if that's the way you prefer to work. My point is, that's not how I prefer to work, and for my purposes, the formatting applied by backticks does matter. I'm not constraining you; you're constraining me. – Marthaª Sep 12 '12 at 20:31
  • @Marthaª OK, I'll buy that – StoneyB Sep 13 '12 at 0:01
2

If keywords will be listed at the bottom of a post, then as StoneyB said, who cares how they're formatted? The current behavior is fine.

If keywords will be embedded in running text -- that is, you're asking people to use <code> to flag keywords in the body of the post -- then you should not change the formatting at all. Keywords are meta-information; you don't need to present that meta-information to readers. Cluttering up a paragraph with monospaced text, over-formatted text, tiny text, weirdly-colored text, etc makes it harder to read. (Yeah yeah, I couldn't exactly implement all those examples right given the limited markup permitted here.)

  • Well, the formatting would be monospaced because in order for it to work, we'd have to use code tags, which currently put that formatting on it. – Kit Z. Fox Sep 11 '12 at 17:49
  • 2
    @KitFox, I read the question as suggesting CSS overrides for code (given that code can mark keywords). Sorry if I misunderstood. <code> with monospace in a line at the end is fine and seems easy for all involved; if the idea is to use <code> inline to mark keywords, I'd want to remove its styling. – Monica Cellio Sep 11 '12 at 17:59
1

Why not treat anything already between quotes as 'searchable'? That is, in the tokenizer throw out stop words as usual except tokenize everything that is already quoted: single, double, italic, bold, code, whatever.

That way, there is no laborious editing that needs to be done by anybody by hand, no need to educate people (which will be long hard and won't benefit the writer), no annoying UI changes. There might need to be a retokenizing of all posts so far, but that is something for the developers to run once.

It seems that the OP is not about a theoretical, to-be-developed feature, but just describing an existing feature (no development at SE needed at all) that allows literal searching on text between backquotes.

My suggestion here is that when analyzing/tokenizing is done, in the character class of 'code delimiters' which currently is

[`],

the developers simply change it to

[`"'].

  • I'm sorry, but this is not on the table. The workaround is specific to existing functionality, which is "stuff between code tags." – Kit Z. Fox Sep 11 '12 at 12:45
  • but code tags are just another kind of quote. instead of "stuff between code tags" just do "stuff between quotes". – Mitch Sep 11 '12 at 15:31
  • My original intent was to pitch the prospect and get a feel for what people thought of leveraging existing functionality - I did in fact expect a fair amount of "no" based on the current aesthetic. I cannot, at the moment, do anything about queueing up a new search parameter, no matter how simple it is in implementation and execution. – Grace Note Sep 11 '12 at 17:33
  • I think the esthetic is irrelevant (or at best a distraction). This is all just education: "If you search with 'code:' then you can find things within backquotes literally (stopwords and punctuation and all). So if you want, start backquoting stuff to allow it to get searched.". Is that the gist of the situation? I.e. this is not a 'hack' or 'workaround' but really, "this is how to do it now." Just that we'd have to start quoting things with backquotes for the benefit of future generations. – Mitch Sep 11 '12 at 17:53
  • KitFox, @GraceNote: if I have it correctly, my answer is actually a new feature request for searching rather than a reply to the OP about CSS styling. Though my answer is thus an indirection, I'll let it stay since I think it responds to the actual need better. – Mitch Sep 11 '12 at 18:10
  • Kinda sorta? It's not just backquotes - using the 4 space indentation to activate markdown code formatting also works (and is largely what other sites who use it more primarily do it). This isn't meant as a "This is how to do it now", rather "This is how you might do it now, if you choose it." Expanding the function of the search parameter to hit different formats, or to hit things which are not formatted at all (double quotes do not apply any special formatting, after all) would indeed be a separate feature request. – Grace Note Sep 11 '12 at 19:13
1

I'm one of those who never uses code markup on ELU because (1) frankly, it looks ugly, and (2) it's pretty well-ingrained on the web that "a bit of typewriter font in the middle of proportional-spaced text" is synonymous with computer code, and is thus not really appropriate on ELU. Apparently, this means that I've been unknowingly disabling the search functions every time I've used bold or italics or quotes to make a use-mention distinction.

So, if we are to start using code markup to highlight keywords, the formatting needs to not scream "I'm computer code", and the first step to that is to use a proportional font, not a monospaced one. It also needs to be differentiated from the existing bold and italic formatting, which pretty much means using color in some manner.

Another consideration is that there are people who have used the code markup on this site, and at least some of them doubtless used it because they wanted their text to look like typewriter-on-gray. So we don't want to totally change the appearance of this text; blinking green is Right OutTM.

Thus, the best formatting option in my opinion would be to use exactly the same font as the rest of ELU, but with a gray background like the existing code formatting.


Edit: I took a look at the html that results from the two types of 'code' markup, and my concerns re:<pre> appear unfounded. Backticks (inline code) apply <code> tags only. Four-space indents (block-level code) apply both <pre> and <code>. This means that it should be easy to achieve the best of both worlds: use <code> to apply a background color only, without changing the font family, and use <pre> to apply the whitespace preservation as well as the monospaced font family. This would mean that people could continue to use code blocks for tabular data, while inline highlights would be much less ugly.


I think relegating keywords to only a section at the end of a post is bad idea. If people want to add a keyword section like that to the end of their posts, that's all well and good, but I don't like to work that much. The way I see it, this searching-in-code business would be most useful if it Just Worked, without any extra effort on anybody's part. And the way that can happen is if we switched to using backticks to mark words that we mention: instead of

Should the following use that or which?

we'd have

Should the following use that or which?

Both serve the purpose of highlighting the words that are important to the question, but only the latter is returned in search results.

(We'd still need to train people not to use code formatting in lieu of blockquotes; most of the results currently returned by "code:[stop word]" searches are irrelevant posts where somebody used code blocks instead of blockquotes.)

  • 2
    The idea (as I understand it) is that it's intended here for additional code words which are used in a similar way to the existing tags. And it would only be needed for "stop" words. So for example at the end of your post there could be Additional keyword: pre – Andrew Leach Sep 11 '12 at 0:05
  • 1
    Code tags are used on EL&U for presenting tabular data. The monospaced font, and the preservation of whitespace, are important factors there. – TRiG Sep 11 '12 at 13:55
  • If you specify a background color you must also specify a text color. It's an accessibility issue. – Monica Cellio Sep 11 '12 at 16:00
  • @Andrew: I like your idea of including an "Additional keyword:" prenote. (I think what you wrote, at the bottom of a question, would be much preferable to pre sitting all by its lonesome.) – J.R. Sep 12 '12 at 9:21
-1

KitFox's question includes several issues that should be discussed in separate meta questions, as noted below; but first, let me suggest we say search-terms (or the gross searchable-terms, etc.) rather than keywords, as the latter is confusingly close to the question-tags concept.

Issues that deserve well-couched questions of their own:

• Whether code-formatting (backtick-delimited or four-space-indented) should be co-opted to mark search-terms
• Visual appearance of user-marked search-terms [which, if OP formatting is the guide, is the main thrust of OP]
• Whether user-marking of search-terms can succeed
• What long-term changes should be made in the SE search engine or search box to better deal with ELU-relevant searching

Regarding that fourth issue, what would be useful to me for searching ELU would be having the top-of-page search box do a Google search instead of an SE search. Conceivably, a user-selected formula could specify a search engine, SE, Google, Bing, etc., but for simplicity let's suppose the user's search text were tacked on at the end of site:english.stackexchange.com/questions and shipped to Google, as is done in the Google box on the search-tips-for-better-search-results webpage. I'd prefer to have -newest -recently also tacked on, as in the following three searches:

• for "“its” and “it's”", about 41 results
• for "“its” and “its”", about 2 results
• for "“it's” and “its”", about 4 results

Regarding that search-tips page, I should mention three problems:

• The [tag] [another-tag] apples oranges example is unclear.
• The code: prefix is not mentioned anywhere on the page.
• The “Advanced Super Ninja Search Options” section should instead be called “Ninja-in-Training Search Options”.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .