Take Control of Your SharePoint Crawl Schedules, Content Sources and Indexes

All my farms were set up (the old way) with one server running Central Admin, and generally I use that for an Indexer too. On top of that, I use a certain utility (which I am not at liberty to discuss in detail) which also runs on the Central Admin server(s). This can sometimes put an intense burden on one server (while the others are loafing).

One thing I am going to do, is load balance my Central Admin and spread it (and that certain utility) around. My setup now is just plain wrong. But in my defense, at one time it was right, and when we did our sanity checks with other experts, it still was right. Once things started getting heavier (indexing 500 GB, then over 700GB of content), there isn't much room for error. Our data center is miles away, but I think I can hear the little fans in those blade servers cranking like mad from here. VVvvvvvv...

Until I redo my Central Admin sites, I have to be cautious about what activities that over-burdened server is doing. One thing I do is turn off indexing when I know I will need those resources. Indexing takes disc space and CPU. I've seen big full crawls fill up lots of temporary space. Be sure to factor this in. It's so easy to take control. You can stop a crawl, but better yet, just don't let a new one start. Change the schedule to fit your activities. Use something like "Every 20 minutes from 8:00 AM for 810 minutes." That will start the last incremental at 9:30 PM, and at 10PM, you should have all the resources of that server for other tasks.

This relates a bit less, but one of my regrets on my setup is that I didn't separate things into different content sources. The logical thing would be to break it up by web app. I will do this soon too. That way you can prioritize (by starting one first) or set separate schedules for different content.

In a really large site, I suppose you could use separate SSPs and manipulate where they are and what they are indexing. We started to go down that path, but if you keep everything together, and just watch how you set up content sources and crawl schedules, you can get a whole lot of useful search indexing (out of not much hardware). Under one SSP, you can have a universal, site-wide index, with no additional setup.

More on AVCHD and Blu-ray (and M2ts)

First, read my first post on the subject. I talk about the utilities I have found that are great for manipulating MKV files and creating an AVCHD file system, suitable for use with Blu-ray players.

For whatever reason, I have found that demuxing into the AVCHD file system format has some overhead - maybe 5-8%. It really shouldn't expand, but for some reason it does. This is enough that sometimes it is difficult to get things into a size that will fit on a DVD 5. I don't have a solution for this other than using some other utility to re-write the file with more compression.

I have found that a 4GB M2ts (MPEG 2 Transport Stream) file will turn into an AVCHD with enough room left to be comfortable - a 100% success rate. Using TXMuxer and MKVExtractGUI I have made some disks that work great.

About audio with AVCHD, like a DVD-Video, you can have a PCM track, but there will only be room for that if the video is compressed like mad, so you pretty much need to resign yourself to (a high bit rate) AC3 5.1. Blu-ray has some wonderful audio capabilities, but to get those, you will need the extra space of a Blu-ray. The AVCHD is limited to the two formats.

Bing vs. Google

A lot of people who saw me using Bing (ever since it came out) would chuckle and make remarks to the effect of having thought higher of me - that I wasn't a true Microsoft bigot. But I am, I give them a chance and the benefit of the doubt every time. They truly are a much more benevolent bunch than ANY of the large influences in the corporate world. But I try to be an open minded bigot. Hey, I got an iPhone (and learned how to make it work MY way), OK. That's my proof.

BING vs. GOOGLE

So Bing vs. Google. Google is an extremely useful tool. For casual users, it gives them what they need very efficiently. For power users, likewise, even better. It's indispensable. Bing comes at things from a completely different perspective. It seems to have a strong commercial slant. I am searching for something so I must want to buy it, or buy a book about it. This is great if I do, but more often I don't.

Bing also seems to have a strong bias to sites closely related to Microsoft and away from sites related to Google. If I search for topics on this blog in Google, they come up very high. Much less so in Bing. On SharePoint things they may not come up at all. That's not right. It could be said that Google has the bias toward their sites (which this is). Probably BOTH are true, but Google seem much better balanced.

Google is still much more useful for my day to day needs, and will be put back into the browsers where it was temporarily replaced by Bing. This will mean that I will Bing very seldom - probably about as often as I used Live - not very (but still more than I ever used Yahoo for anything - how could that mess be useful to anyone).

Bob Dylan Show Comes to Eastlake

On Saturday, July 11, 2009 Bob Dylan, John Mellencamp and Willie Nelson brought their tour to the Captains minor league ballpark, Classic Park, in Eastlake, Ohio as a part of their tour of minor league baseball stadiums. For your $75 (including fees), they bring along a barely adequate (not good) sound system. The stage was decent. There are lights, but not what I would call a light show, and there is no TV. They are counting on the music to entertain you.

But the vibe was still nice, as were the performances. They had a redeeming gimmick - children 14 and under were free.

The whole show people were talking to me. They were saying, "C'mon Bob" and stuff. Wrong Bob? This stadium is about 2 miles from my house (in a straight line) so I felt very at home. It was nice to have Bob over so he could jam on his organ (unfortunately, he only played guitar for two songs).

Set list:

Cat's In The Well

Señor (Tales Of Yankee Power)

Rollin' And Tumblin'

Blind Willie McTell

Tweedle Dee & Tweedle Dum

If You Ever Go To Houston

Things Have Changed

Workingman's Blues #2

Highway 61 Revisited

Ain't Talkin'

Thunder On The Mountain

Like A Rolling Stone

Jolene

All Along The Watchtower

Edirol FA-101 Windows 7 64 Bit Drivers

The original post I had here was a work around. Roland published Windows 7 64 bit drivers in late 2009 and I have found them to work well with Windows 7. Go to Roland and get them.

If anyone has tried these drivers with Windows Server 2008 R2, please let me know. Thanks.

You really don't need the following information, but I leave it here for historical purposes.

Original Post:

Windows 7 again impressed me when I needed to load up my FA-101. I use the FA-101 as the only audio interface on my studio PC.

Doing a quick search you will find that there are no Windows 7 64 bit drivers for this device and if you try to run the Vista installer (FA_101_Vista_x64_Ver.1.0.4) you will get an error that it will not run for this version of Windows.

All you have to do is run Windows 7's (very cool) "Troubleshoot compatibility" wizard.

You select "Troubleshoot program" and check the "The program worked in earlier versions of Windows ..." box. Indicate that is was Vista, and then the driver installs as it did in Vista.

I have found the FA-101 to be much less susceptible to problems under Windows 7 than under Vista. Although the FA-101 driver did not contribute to my former 64 bit Vista's instability, the driver itself would frequently seem to get stuck. I was very used to powering off the FA-101 to un-stick it. So far, that has not been necessary with Windows 7. It continues to shine!

Windows 7 Amazing Installation

I lost one of the two 400 GB Western Digital drives in my RAID 0 array on my home "studio" PC on Sunday. The performance of two mediocre drives raided together has always been impressive, so I was determined to go back to it. This desktop machine was running 64 bit Vista, and was not particularly stable. I was hoping to put either Windows 7 or Server 2008 R2 on it, and I was going to wait until fall.

Monday I bought two Seagate ST31000333AS 1TB drives from Antares Computing which arrived yesterday. I recommend both Antares and these drives (both are fast!). I only recommend RAID 0 if you can deal with losing the array (as I could). Reliability will be low - performance high.

So I take out the two old drives and stick in the two new ones. It takes longer than it should - thirty minutes - a little bit of cable manipulation and contortion (I never remove the full tower case from the rat's nest where it lives). Next I stick in the Windows 7 RC disk and boot. I think I had two clicks before it took off, and 20 minutes later it was done.

This bears repeating. Twenty minutes later, Windows was installed into previously unallocated space on freshly installed hard drives. I am sure I will have a lot of playing around to do (but I enjoy that - hit me). Twenty minutes. No formatting. Nothing.

Fixing SharePoint Search Propagation Problems - Part II

As a continuation of Fixing SharePoint Search Propagation Problems ...

In my previously described situation (stuck on propagation with a few other issues thrown in), the problem recurred, and repeating the procedure (moving the index file location on all query servers) again provided relief.

Since I needed more than a few hours of relief and I could see nothing new was making it's becoming available to search, I also stopped indexing. Over night, this allowed propagation to clear itself. Since all was well, I boldly turned indexing back on. More good results - all the new content made it to the query servers and continued to do so. I still may have issues because after a while, the propagating status once again seemed stuck, but content is getting added and we are otherwise healthy (otherwise - that is a big qualification).

It would be great to get an in-depth explanation of what the services are doing when they report various statuses. I wish I could give one - I want to get this explanation. My own experience has been that although monitoring the services can be alarming, they work well. Indexes are built and updated quickly, even when things do not appear to be working very well. This from a person whose life has been made much more stressful by having to watch over this system.

Using MKV Files With a Memorex Blu-Ray Player

I naively bought a two Memorex MVBD 2510 players last fall - one for my dad and and one for myself. They were good Black Friday deals, so I have no buyer's remorse. I knew they didn't have an ethernet port, but I had no idea what good it might be. I still can only guess what utility I might get from having my player on my home network.

Like most of these players, it has a USB port, so I quickly got into experimentation and found that it would play MP4, AVI, and a mess of other file formats, but not every file format. Around the same time, MKV files started to become prevalent. The Memorex doesn't like MKV's but they play wonderfully from VLC on my computer (which has a 24" HD widescreen). I don't live and breath for this stuff, but I am an enthusiast, so it took me a while to understand why we needed yet another format!

MKV (Matroska) is a container. From what I can tell, it is lossless, and it gets a small amount of compression. They can hold a lot of stuff, but in order to use them with devices that don't support it (and aren't being well supported by the manufacturer), you have to do something with them. I have the answer for that!

You could get a conversion program that knows how to read these files and write them out to another format - like AVI or MP4. That means all your content is going through another rendering and compressing process. Although the results may be acceptable , I know there is no way doing this is going to improve the video or audio. And it takes a long time. We need something better.

The answer is demuxing the file (and maybe doing something cool with the demuxed content). Demuxing the MKV contents to a .ts file will work with the Memorex (and thus I would think many other - better - players). The tool to do this is TXMUXER. Demuxing is as fast as your disk (you will wish you had a fast disk setup) and there is no re-compression.

The only problem is (at least with the Memorex) that .ts files act a lot like AVI's or other files played on the Memorex - rewind, fast forward and other operations don't always work as you would like, but otherwise, it's not a bad arrangement. You get the nice menu from the player and you can select the files nicely (just don't make the name too long - no scrolling on the Memorex).

MKVExtractGUI includes the ability to extract files to a Blu-Ray or AVCHD folder structure that it creates. AVCHD is what you want. Many BD players (Memorex and Playstation 3 included) support AVCHD. It's basically a Blu-Ray file system for DVD 5 (4.7GB DVD recordable discs). I haven't explored all the audio possibilities of this (Blu-Ray includes several outstanding lossless formats). These AVCHD discs have no menu, but you can "join" multiple input files and add chapter stops at whatever point you like (think "episodes"). Just burn the two folders the process creates to a UDF file system on a DVD 5 (I have not yet tried this with a DVD 9, but that is coming).

With a little trial and error, I've come up with a nice way to play and store content.

Fixing SharePoint Search Propagation Problems

See also Part II

I have been baby sitting my SharePoint search engine lately. I have some long term fixes I am planning to deploy, but in the mean time, things sometimes go bump!

The latest problems surround the engine sticking on "Computing ranking" and then today a propagation issue. The indexer eventually found itself unable to communicate with the query servers. This got ugly fast. Search results turned to garbage. They would cross scopes and return all kinds of erroneous stuff.

I was getting pannicky, all the while hoping I would not have to reset content. This I thought would very likely be a total fix, but drastic - the all new index would be pretty meaningless for quite a few hours. Resets of services and things were not going to fix this problem. Since the indexer seemed to have lost contact to the shares where the indexes were supposed to go, I decided to change the locations for the index files on the query servers.

In case you forgot, you do this in Operations > Services on Server > Office SharePoint Server Search Service Settings. For each of them, I put it in a brand new location. This didn't seem to help at all until they were all done. Then it all came back (fairly quickly). I still have a stuck on "Computing ranking" problem, but I have more reasonable content. I still intend to do that "Reset all crawled content," but I can do it in off hours.

See also Part II

PDF Ifilters in SharePoint - Follow Up

I would like to add an addendum to http://bobklass.blogspot.com/2009/04/pdf-ifilters-in-sharepoint.html.

I was not aware of PDFLib's ifilter until I was near the end of my testing and procurement a few weeks back. Since I still had a pretty good test setup for this, and I needed to do some final QA, I decided to test PDFLib too.

Before you can test PDFLib you will need to get yourself a license from their sales group. They give you a reasonable testing period and the process is painless.

In today's test, PDFLib's installer didn't add both of the registry keys I mentioned in my last post, only the first.

• [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\12.0\Search\Setup\ContentIndexCommon\Filters\Extension\.pdf]
  
• [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0\Search\Setup\ContentIndexCommon\Filters\Extension\.pdf]

The GUID for PDFLib seems to be {47A1AF35-C345-475d-AE68-EB07E948BD07}

I added the second manually. I would guess it is not important if you have MOSS.

My test set was not a very challenging bunch of documents (about 680 MB, 1200 documents). In some previous tests, Adobe was much slower (as anticipated) but in the ones today, they all three were similar. I previously was running the test after resetting the content. On today's I did full crawls (to save time??).

Interestingly, the quality of Adobe was much lower. The same search terms produced far fewer hits with Adobe while Foxit and PDFLit had the exact same results. It almost seemed anomalous, but my main objective today was to review the installation process (I have beaten that to death, no?) and give PDFLib a quick test, so I don't have time for further tests.

We already bought Foxit. I may have been able to save a little money with the way PDFLib is licensed, but I think I made the right choice because of Foxit's more foolproof installer. Hopefully I can add one more follow up article once I have been running Foxit in a large production environment for a while.

Eric Brown - Kethani

English author Eric Brown's 2008 sci-fi novel Kéthani shows an earthly world that is at once scary and intriguing. An alien race has left its' imprint on our society by giving it a boost to the future. The characters' lives are described by their central point of reference - the pub. We readers get to meet them at "the Fleece" for a few pints every Tuesday (and many other days that end in "ay").

Unfortunately, as a reader, I cannot be "implanted" and so the association must end. But I thoroughly enjoyed the company of so many of Brown's lifelike characters. So much so that next I will read his Helix (unrelated story).

Highly recommended: Eric Brown's Kéthani.

Nancy Kress - Steal Across the Sky

A recent good and fast read of mine, Nancy Kress' Steal Across the Sky is a great romp through a near future world where we have made alien contact. These aliens had been here before, and they messed things up for us. But they came to make things right and thus they called themselves "the atoners." They send witnesses to see what they messed up and ... well I don't want to spoil it.

I do want to say that Nancy Kress is a great find for me. My taste lately has been to seek out new books and try new-to-me authors (hey, Robert Silverberg and some of my other old favorites aren't churning them out the way they used to). She reads very smoothly - crawls right into your head. There were some global warming related story points that are based on bad science, but like other bad sci-fi inventions, you can just look past that.

Nancy Kress is a celebrated author and with this, her latest, it is easy to see why.

... and sorry about the recent "test" post, everybody. We were trying the Word 2007 integration on our (new, not yet completed) standard desktop, only to find that it can publish to Blogger rather nicely if you know what you are doing (or almost do).

Cedar Point 2009

What can I say about Cedar Point that I haven't already said? It is one of my favorite places, and I'll bet you would love it too. Yesterday we had a wonderful day. Perfect weather. Lots of great rides. The Dragster picture looks just like one I posted before (only a year older). The rest of the pictures (click) are seriously sun-drenched.

From Cedar Point 2009

We had one big mishap, and it paid some dividends. Early in the morning we were going to ride Maverick, which I had yet to ride. There were very short lines, but while we were waiting, it had a minor problem. A few minutes later they had it back up and we were riding. Maverick has two lift launches. On the second launch (which is out of a dark tunnel) we shot out as designed, but at the top of the hill the machine braked heavily and we slid back down backwards, all the way into the tunnel. There we sat for 40 minutes - 12 of us. Harnessed in with no way out.

Eventually they got the machine working and we were re-launched. When we arrived in the station they gave us a free no-waiting pass onto Top Thrill Dragster, Raptor and Millennium Force. We turned the Dragster and Raptor ones into instant front seat rides. Not too shabby.

In general, Cedar Point never looked better. The place is extremely clean and well manicured. Everything (well, most things) seemed to be in great working order. Even the ticket prices were reasonable (early season Internet specials). Famous Dave's was great, as always, for BBQ wings and Sam Adams. I hope the coming Memorial Day weekend turns out as nice.

Here's a tip:
save your parking receipt and turn it in for 100% credit at Famous Dave's. I've been doing this for years. $10 off! It works at some of the other places on the pennisula that aren't in the main park, but you won't find it printed anywhere.

PDF Ifilters in SharePoint

See also http://bobklass.blogspot.com/2009/06/pdf-ifilters-in-sharepoint-follow-up.html

We have more PDF files than anything else in our SharePoint sites, by a huge margin. We also have some pretty over-sized hardware, so up until recently, we have been able to get by with the single-threaded free Adobe ifilter.

When the Foxit ifilter was released I did some testing and could not see sufficient benefit to make the change. I am pretty sure my testing was flawed, as others have confirmed the superior performance of the Foxit ifilter. I carefully retested it and with a relatively small sample size (less than 2000 files) I got a benefit of not quite double the throughput. I think that with a larger sample the benefit would be greater, but the successful test is sufficient for my needs and allows me to justify this purchase.

Checking Ifilter Registry Entries

Along the way I experienced some difficulties. The instructions for installing the Foxit ifilter are very simple, and sometimes they work as they should, but not always. If you are doing a first time installation of a PDF ifilter, follow Foxit's instructions carefully. For a replacement of Adobe, you simply uninstall the Adobe ifilter and then you install Foxit. I did this uninstall/install cycle a couple times. Either of the uninstalls may not always happen cleanly, so you may need to get out your SharePoint/IIS/Windows hammer and do a bit of tapping.

As I was testing, I was getting an error on all the PDF's that there was no ifilter installed. I ended up manually modifying two registry keys, at first just to put back Adobe so I could test that. Then the uninstall/reinstall worked, but I still checked it just to be sure. My problems could have been from not following procedures (starts, stops, etc.) or possibly from the install / uninstall not being complete. Checking these keys was a big part of my solution.

The two keys are:

• [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\12.0\Search\Setup\ContentIndexCommon\Filters\Extension\.pdf]
  
• [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\12.0\Search\Setup\ContentIndexCommon\Filters\Extension\.pdf]
  

Their multi-string value should be the appropriate GUID for the classid for the ifilter you are using (I think that's what this is):

• {987f8d1a-26e6-4554-b007-6b20e2680632} Foxit
  
• {4C904448-74A9-11D0-AF6E-00C04FD8DC02} Adobe 6
  
• {E8978DA6-047F-4E3D-9C78-CDBE46041603} Adobe 8 (or 9 or 64 bit??)
  

You also need to be sure to start and stop the search services and why not throw in an IIS reset while you are at it (hammer). Of course a reboot has the same effect (more hammering). After doing that, to make your tests cleaner, it's good to get the setup primed. You can do that by running it twice (reset the content in between), or just enough the first time to be sure that everything is running. If you monitor the index you will see a minute or two delay on an un-primed setup.

I am looking forward to getting this into a production environment with hundreds of thousands of PDFs. I hope I won't need my hammers, but at least I know where to look if it doesn't seem to be working.

See also http://bobklass.blogspot.com/2009/06/pdf-ifilters-in-sharepoint-follow-up.html

Jeff Beck Cleveland Set List

In response to a request for a Cleveland (April 15, 2009) Jeff Beck at the House of Blues set list:

1 Beck's Bolero 
2 The Pump 
3 You Never Know 
4 Cause We've Ended as Lovers 
5 Behind the Veil 
6 Blast from the East 
7 Stratus 
8 Angel 
9 Drums 
10 Led Boots 
11 Nadia 
12 Space Boogie 
13 Goodbye Porkpie Hat 
14 Blue Wind 
15 Bass 
16 Day in the Life 
17 Big Block 
Encore Break 
18 Where Were you 
19 Scottish One 
20 Theme from Peter Gunn 

Tal Wilkenfeld-Bass 
Vinnie Colaiuta-Drums 
Jason Rebello-Keys 
Jeff Beck-Guitar 

You can probably find an active torrent of this show. I have heard two very similar sounding recordings (OK, but A- grade). 

The show was a nice display of Jeff's guitar prowess and thoroughly enjoyable. A sonic treat. I was very disappointed that the "Jeff" album was not represented (I hoped it would be featured). Jeff's last three albums have been explorations in this direction, with Jeff being the pinnacle of the bunch. 

Here's the set list for the show I am seeing tonight:

;<)

Cleveland Orchestra - Sir Colin Davis, guest conductor

Hector Berlioz - Overture to Beatrice and Benedict

W.A. Mozart Piano Concerto No. 25 in C major, K503

1. Allegro maestoso

2. Andante

3. Allegretto

Mitsuko Uchida, piano

INTERMISSION

Jean Sibelius Symphony No. 2 in D major, Opus 43

1. Allegretto

2. Andante, ma rubato

3. Vivacissimo

4. Finale: Allegro moderato

Travis Review

This is the text from my gushing review I posted yesterday on TravisOnline.com:

What a lovely, lovely show. All day yesterday I had the Monday blahs. I was wondering how I could get myself up for seeing Travis. I listened to a lot of Travis during the day, and that helped, but it wasn't until Travis started playing that my day turned around. And it really turned around.

As the show progressed, you got the feeling that everyone was having the same beautiful experience. We were all feeling the power of the music. I had seen Travis once before in Detroit, several years ago, and it was the same experience, but on a slightly larger scale. The House of Blues crowd was less than capacity (it was a rainy, crappy Monday night) but that added to the intimacy of the show.

Having checked all the set lists posted on this site, I noticed that "As You Are" was missing. I was sad about that. You could feel that this was a really good show the band was playing - one with lots of positive feelings. When they played "As You Are," it went over the top for me. 

It was a very special Monday. Thank you Travis.

Jeff Beck, Travis, Berlioz, Mozart and Sibelius in 8 Days

Today begins a very full 8 day week of music. I have been looking forward to tonight's Jeff Beck concert at the Cleveland House of Blues for some time. Years, in fact.  Fortunately, Jeff Beck has been touring a lot lately. He fronts a band with the highest caliber of musicianship. 

Next Monday is the great brit-pop group, Travis, also at the Cleveland House of Blues. Travis latest album is similar to their other hauntingly dreary yet uplifting albums, but more electric. 

I am looking forward to seeing both of those shows from the extreme comfort and sonic heaven of the HOB balcony. 

Also a sonic heaven will be the Cleveland Orchestra with guest conductor, Sir Colin Davis next Thursday. Cleveland's Severance Hall is a wonderful place. My employer is sponsoring these concerts, so heavily discounted ($60 tickets for $20) were available. 

Speaking of ticket prices, the Jeff Beck tickets were the most expensive I have purchased for the HOB at $99 each, including the ridiculous fees. You may or may not know, I used to boycott the HOB because they are an out of town company that received tax abatements so they could put local venues out of business. Once I broke my boycott, the flood gates opened, but I still bad- mouth it and the bar stinks (high prices and poor selection).

Aloha

Greetings from Hawaii. What a crazy long way from home this is, but how simple it is to get here (put it into your favorite mapping program, it will tell you to just go straight across the Pacific - not a problem, but a jet helps). It's just another state. So everything works, U.S. money, bank cards, cell phones. The Outrigger Waikiki Beach hotel where we are staying is extremely hospitable. This should be a fun visit.

Live Recording with the Zoom H2 - Epilog

See previous

Last Saturday I took the H2 to the House of Blues in Cleveland and recorded Dickie Betts. Dickie allows taping, but I thought I'd try out my stealth since they have overly strong security at that venue. I had no issues with stealthiness.

In general, I would say that recoding with the H2 is much less prone to error than with the Rockbox iRiver. Once it is set, just turn it on, press record to arm and press record again to confirm. I used the same levels I used for the last show (78, and it is set at the least sensitive level).

I have not used any of the compression or limiting and there hasn't been any issues with overload. The levels also haven't been too low. Just no issues (so far).

The recording is great. Betts brings a three guitar, two drummer (plus bass and keys) ensemble. They can do "Allman Brothers" style music very effectively, and as my friend Nick said, he obviously just loves to play.

How Bad Is it?

8.1% unemployment. That's bad. Unemployment was much worse not that long ago. How about the entire period of November 1981 through December 1983. What was that like? In the media it was always somewhat gloomy, but it seems like the media negativism is worse this time. And even more mindless.

There have been some economic extremes in my lifetime. How about 18% mortgages? Or double digit inflation year after year. Globally. And whatever happens to the U.S. is usually much worse in most of the rest of the world.

Most people in the U.S. do well all the time. Some never do well. During such crises, there is a bit of crossover of the mosts to the somes. If the media didn't over play it (and we "mosts" never looked at our 401k statements), the mosts would hardly be aware of the crisis. 

I truly believe that contributions from the media towards a negative psychological state of the masses are significant to overall trends. Most of it isn't news from a well-informed source, it's just drivel. 

So the answer is, it's not that bad. Who do you trust, TV writers or me? At least I have an MA in economics. If you have some dough and can figure out some winners, it's time to buy!