jList: Performance improvements when removing duplicates

Version 1.2.0 of jList brings performance improvements to the two most recently added functions listRemoveDuplicates and listRemoveDuplicatesNoCase. These now run about 30% faster than before though you are unlikely to notice the gains unless you are working with very large lists (100,000+ elements) or running it many times consecutively.

I intend to post later this week about efficiently removing duplicates from Javascript arrays so I shall cover what changes I made in more detail as part of that post.



jList: Two new functions to dedupe lists in Javascript

Earlier today I released version 1.1.0 of the jList Javascript library. This release adds two new functions, bringing the total to 15.

The new functions are:

  • listRemoveDuplicates
  • listRemoveDuplicatesNoCase

As you might guess from the names these are intended to remove duplicate elements from lists.

A simple code snippet illustrates the difference between the two:

mylist = 'cat,dog,monkey,DOG,Dog,cat';

response = jList.listRemoveDuplicates(mylist);
console.log (response);

response = jList.listRemoveDuplicatesNoCase(mylist);
console.log (response);

In line 3 we perform case-sensitive matching when looking for duplicates. As a result, we don’t consider ‘dog’ to be the equivalent of ‘DOG’ or ‘Dog’ so the result displayed in line 4 will be ‘cat,dog,monkey,DOG,Dog’ – only the duplicate ‘cat’ has been removed.

In line 4 we perform case-insensitive matching when looking for duplicates. As a result, the output from line 7 will be ‘cat,dog,monkey’ as ‘dog’, ‘DOG’ and ‘dog’ are all considered equivalent in this case.

Notice also that in both functions the order of the list elements is unchanged. This illustrates that it’s the second and subsequent occurrences of an element that are considered to be the duplicates and removed, the first occurrence remains in its original place within the list.

The new version of jList, including updated documentation, is available for download from GitHub.


jList: ColdFusion-style list handling in Javascript

jList is a collection of functions that I wrote some time ago to bring ColdFusion-style list handling to Javascript. A list in this case is simply a string. What makes this string different (from any other string) is that it contains “delimiters”. Delimiters (sometimes referred to as “separators”) are used to separate each item in the list.

For instance, the string “carrots,peas,lettuce,tomato” could be described as a “comma delimited” or “comma separated” list. Although commas are the most common delimiter for lists, any character can be used – “Monkey:Rabbit:Mouse:Elephant” is no less a list than the comma delimited list.

Once you start to use jList you will quickly realise that there are many situations where treating a string as a list is a simpler and more intuitive way to manipulate data than the built-in alternatives offered by Javascript.

There are 13 functions included in the library. They are, in alphabetical order:

  • listAppend
    Concatenates an element to a list.
  • listDeleteAt
    Deletes an element from a list.
  • listFind
    Determines the index of the first list element in which a specified value occurs. The search is case-sensitive.
  • listFindNoCase
    Determines the index of the first list element in which a specified value occurs. The search is case-insensitive.
  • listFirst
    Gets the first element of a list.
  • listGetAt
    Gets a list element at a specified position.
  • listInsertAt
    Inserts an element into a list.
  • listLast
    Gets the last element of a list.
  • listLen
    Determines the number of elements in a list.
  • listPrepend
    Inserts an element at the beginning of a list.
  • listQualify
    Inserts a string at the beginning and end of list elements.
  • listRest
    Gets a list, without its first element.
  • listReverse
    Reverses the order of elements in a list.
  • listSetAt
    Replaces the contents of a list element.
  • listSort
    Sorts list elements according to a sort type and sort order.

Once the library is loaded then invoking the function you want is simply a matter of prefixing the function with “jList.”. The following example, which uses the listAppend and listSort functions, shows how:

<script src="jlist-min.js"></script>
    var l, l2;
    l = "Susan,Janet,Steve,Bree,Michele,Jim,Mark,Elizabeth";
    l = jList.listAppend(l, "David");
    l2 = jList.listSort(l);

This will show an alert with the text “Bree,David,Elizabeth,Janet,Jim,Mark,Michele,Steve,Susan”.

List processing is convenient for handling many strings that don’t at first appear to be lists. For example, the URL “http://www.mysite.net/main.htm?id=1234&#8221; can be considered to be a list or, more usefully, a series of lists. To extract the top-level domain (tld) part (in this case the “net”) we could think of the URL as a “dot delimited” list, within a “forward-slash delimited” list.
var url = "http://www.mysite.net/main.htm?id=1234";
var url_part = jList.listGetAt(url,3,"/"); // returns 'www.mysite.net'
var tld = jList.listLast(url_part,"."); // returns 'net'

In line 2 we grab the 3rd item in the list, using a delimiter of “/”. In this case this is “www.mysite.net” because the first item is “http:” and the second item is empty. This is an important difference between this library and ColdFusion list processing – empty elements are not ignored in these functions. Notice also that we start counting the items from 1, not from zero.
Once we have the third item stored in url_part (line 2) we can then treat this as a “dot delimited” list. Using the listLast function (line 3) to retrieve the rightmost item in the list but this time specifying “.” as the delimiter will return “net” into the tld variable.

For this 1.0.0 release I have cleaned up the code, added detailed documentation and made it public on GitHub.