Processing Lists of Words for Wikipedia

This request is from our member Calaka:
------------------------------------------------

So there is this website (hxxp: www. globalnames. org/) which lists well over 18 million names; many of which are individual genera and species. The problem is that in addition to the legitimate names, there is also a lot of false positives (as the list is gathered with a bot from a number of website resources. So I am a volunteer over at Wikipedia doing minor things, and I happened to start up a little "project" where I gather the list of names for other users (and me) to create articles based on the names provided from that massive list (and utilizing the references they give). That little project page/list can be found here: hxxp: en. wikipedia. org/wiki/Wikipedia:WikiProject_Missing_encyclopedic_articles/Global_Names_Index . Now I managed to get help from another Wikipedian volunteer that used a program with Python to extract the names and for me to then place on wikipedia lists manually (see: hxxp: www. eecis. udel. edu/~mpellegr/wikipedia/data/ . Now he told me that I am all on my own on putting those lists on Wikipedia (as he is not aware/capable of doing the below) for others to create. I did the first 10 pages of A and the first page of B (as an example shows here: hxxp: en. wikipedia. org/wiki/Wikipedia:WikiProject_Missing_encyclopedic_articles/Global_Names_Index/A ).

Now there are two problems to this. First of all, there seems to be a limit of about ~2000 names per list, as after 2,000 the {{search}} function next to the name on the list is no longer functioning, which means there would need to be over 9,000 pages of lists created to cater for all the 18 million names! It would take forever to do by hand (quite time consuming)

Second (which a solution to this would help solve the first problem), there needs to be a way of "getting rid of the crud from the good stuff" so to speak, i. e. removing all the names on that list that are not needed and only keeping those that can be made into an article. Then you might be wondering what is useful and what is not?

Well:

1. Anything that is just one word can remain on the list. (This is a genus)
2. Anything that is two words with the second word being a lower case can remain on the list. (This is a species)
3. Anything that is three words with the second and third word being a lower case can remain on the list. (A lot rarer: this is a subspecies)

Things that can be removed/are not needed to be created as actual articles:

1. Two words in a name with the second letter having a capital letter (which is usually the person who discovered/names the species/genus) - genus if it is two words, species if it is three words (the third word being a capital letter in this case)
2. Any names that have a number next to it (usually coincides with point 1 but not always)
3. Anything with a comma or parenthesis or square bracket

So as you can see. . . If it was possible to design a bot/program to run through that first list made by the guy, compile its own list by following the above 6 points, the list can be reduced to include only those names which are relevant and greatly reduce the amount of names (I estimate from 18 million down to 3-4 million, if not even less!). With that new list present, it would then be possible to paste those in Wikipedia (if you are unsure of how to do that automatically, then surely there would be someone who would know how to within the community - it was just difficult finding someone capable of creating the above program with the above set of rules). The list can start alphabetically with everything below: "Abacopteris penangiana" (as that is the last legitimate species on page 1) and pages 2-10 of the letter A can be redone to correspond with the new rules since no one has gone through them yet, whereas I have been working on page one for a bit now.

Now I know this might seem a bit full on and even more so for someone that does not know with the "behind the scenes" dealings with Wikipedia but if you can advise/give any comments to this or perhaps even recommend a place to post the above, it would be much appreciated.

The code used by the first individual to create that list in case you are curious:

hxxp: www. eecis. udel. edu/~mpellegr/wikipedia/gen

Oh and PPS: The names themeselves can be created into articles by following the references/lists provided by the global names index. An example of such a page can be found here: hxxp: en. wikipedia. org/wiki/Asceles and here: hxxp: en. wikipedia. org/wiki/Akera_bullata (I made a few more, but I think you get the idea of what is possible with the references provided)

		Email this topic \| Print
0 Members and 1 Guest are viewing this topic.